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EXECUTIVE  SUMMARY 


This  project  is  part  of  an  overall  effort  of  the  Federal  Aviation  Administration  (FAA)  to  ensure 
that  airplane  simulator  fidelity  is  commensurate  with  its  current  use  for  total  training  and 
evaluation  of  airline  pilots  in  the  flight  simulator.  To  assess  the  efficacy  of  current  radio 
communications  simulation  and  to  define  the  requirement  and  feasibility  of  simulating  radio 
communications,  research  into  radio  communications  simulation  was  initiated.  This  report 
summarizes  the  first-year  effort,  including  1)  a  multi-disciplinary  literature  review  focusing  on 
Advanced  Qualification  Program  (AQP)  and  crew  resource  management/cockpit  task 
management  (CRM/CTM);  2)  a  survey  of  instructor/evaluators  (IEs)  to  determine  current 
practices  and  to  assess  their  opinions  regarding  the  impact  of  realistic  radio  communications  on 
training  effectiveness;  and  3)  a  review  of  the  enabling  technologies  that  might  support  future 
automated  simulation  of  realistic  radio  communications. 

Simulators  used  for  total  training  and  evaluation  of  airline  pilots  must  satisfy  stringent  criteria  to 
assure  their  adequacy  for  training  and  checking  maneuvers.  Radio  communications,  however,  are 
currently  not  a  part  of  these  criteria,  despite  their  integral  role  in  every  aspect  of  the  flight 
environment.  Ideally,  simulation  of  air  traffic  control  (ATC)  communications  to  own  and  other 
vehicles  (the  so-called  partyline)  as  well  as  company  voice  communications  should: 

■  be  appropriate  for  the  airspace  and  responsive  to  crew  actions; 

■  reflect  the  demands  of  ATC-driven  timing  that  does  not  take  pilots’  activities  or  workload 
into  account; 

■  contain  a  meaningful  partyline  that  can  be  used  for  enhancing  situation  awareness;  and 

■  expose  pilots  to  communication  problems  such  as  communication  frequency  congestion, 
blocked  frequencies  due  to  stuck  microphones,  or  non-native  English-speaking  ATC. 

The  underlying  premise  of  this  research  is  that  providing  a  realistic  radio  communications 
environment  would  have  a  significant  impact  on  safety  by  enhancing  both  pilot  training  and 
evaluation.  A  realistic  radio  communications  environment  should  provide  pilots  with 
opportunities  to: 

■  practice  frequency  monitoring  as  well  as  task  and  distraction  management  skills; 

■  enhance  crew  resource  management  skills; 

practice  coordination  with  ATC  during  new  procedures,  such  as  landing  on  parallel  runways 
which  requires  an  additional  controller; 

■  improve  their  ability  to  increase  situation  awareness  from  meaningful  partyline  containing 

information  on  traffic  and  weather;  ° 

■  ensure  their  currency  in  ATC  communication  procedures  and  phraseology; 

■  reinforce  attention  allocation  and  differentiation  skills  necessary  to  distinguish  clearances  for 
one’s  own  airplane  from  clearances  for  other  aircraft; 

■  receive  more  focused  attention  from  the  instructor/evaluator; 

■  avoid  an  impoverished  training  environment  potentially  leading  to  the  development  of  tunnel 
vision;  and 
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■  reduce  the  so-called  “simulator  mindset,”  which  may  interfere  with  transfer  of  pilot 
performance  and  behavior  to  and  from  the  airplane. 

LITERATURE  REVIEW 

A  review  of  the  relevant  AQP  and  CRM  literature  underscores  the  importance  of  realistic 
scenarios,  and  that  coordination  with  company  and  ATC  is  an  integral  part  of  both  the  AQP  task 
list  and  CRM.  Additionally,  a  number  of  Aviation  Safety  Reporting  System  (ASRS)  reports, 
research  investigations,  and  industry,  military,  and  general  aviation  magazine  articles  show  that 
radio  communications  directly  or  indirectly  contribute  to  incidents  and  accidents.  These  reports 
underscore  the  need  to  train  pilots  to  practice  how  to  recognize  and  question  inadequate  or 
demanding  ATC  instructions  such  as  last-minute  runway  changes.  The  need  to  practice  correct 
phraseology,  especially  during  emergencies  or  with  non-native  English-speaking  ATC  is  also 
evident  from  these  studies.  The  requirement  for  developing  and  evaluating  CRM  skills  is 
repeatedly  emphasized  in  these  reports,  skills  which  cannot  be  adequately  trained  or  assessed  if 
key  elements  of  the  CRM  scenarios,  such  as  radio  communications,  are  absent. 

INSTRUCTOR/EVALUATOR  SURVEY 

The  Instructor/Evaluator  Survey  polled  29  instructor/evaluators  from  14  different  airlines,  all 
participating  in  AQP.  All  I/Es  queried  indicated  simulating  some  ATC  communications, 
especially  in  the  terminal  environment.  For  company  communications,  all  but  one  of  the  I/Es 
reported  simulating  at  least  some  of  them,  most  frequently  dispatch.  With  regard  to  the  partyline, 
only  38  percent  provided  any  communications  to  or  from  other  traffic,  mainly  on  the  surface. 
Only  two  of  these  provided  terminal  and  en-route  partyline  information.  Most  radio 
communications  are  simulated  by  I/E  role-play,  with  few  exceptions.  Five  I/Es  indicated  using 
handouts  or  ACARS  (Aircraft  Communications  Addressing  and  Reporting  System)  for 
Clearance  Delivery,  and  over  half  reportedly  use  either  printed  or  recorded  (synthetic  or  natural 
voice)  ATIS  (Automatic  Terminal  Information  Service). 

Regarding  workload  in  the  simulator,  I/Es  reported  that  for  full  mission  simulation  training  and 
evaluation  events,  radio  communications  simulation  and  managing  the  simulator  systems  each 
consume  about  20  percent  of  their  time  and  effort.  At  least  half  of  their  time  and  effort  is  spent 
observing.  When  training,  they  spend  8  percent  of  their  time/effort  instructing,  versus  only  2 
percent  when  evaluating.  Most  felt  that  I/E  impersonation  of  communications  to  own  airplane 
increases  the  instructor  workload  and  “divides  his  attention.” 

On  the  other  hand,  most  instructors  indicated  that  the  current  method  of  simulating  radio 
communications  produces  an  unrealistically  low  communications  load  for  pilots  compared  to  the 
real  world,  especially  with  regard  to  company  communications.  One  mentioned  that  I/E 
impersonation  of  radio  communications  even  reduces  the  manual  workload  of  pilots,  because 
they  normally  do  not  need  to  redial  a  new  frequency  to  communicate  in  the  simulator.  Also,  the 
I/Es  think  that  the  presence  of  meaningful  partyline  information  would  increase  training 
effectiveness,  by  “teaching  pilots  to  listen”  and  “enabling  [I/Es]  to  assess  CRM  elements  such  as 
workload  and  distraction.” 
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I/Es  indicated  that  scenarios  rely  on  radio  communications  simulation  to  effectively  teach  such 
skills  as  (new)  ATC  procedures,  CRM,  and  situation  awareness.  One  I/E  alluded  to  the  fact  that 
I/E  role-play  may  lead  to  uneven  training  and  unfair  evaluation,  since  effective  radio 
communications  simulation  is  “all  relative  to  the  creative  ability  of  the  instructor.” 

Although  some  I/Es  were  skeptical  with  respect  to  the  success  of  automating  radio 
communications  and  its  effect  on  reducing  workload,  they  nevertheless  indicated  that  it  would 
increase  training  effectiveness.  Perhaps  due  to  I/Es’  skepticism  regarding  other  kinds  of 
communication  automation,  ACARS  and  ATIS  received  the  highest  rankings  for  the  anticipated 
beneficial  effect  of  automation  on  training  effectiveness. 

TECHNOLOGY  REVIEW 

The  technology  review  included: 


■  United  Airlines’  Interactive  Real  Time  Audio  System  (IRAS),  also  (and  too  dismissively) 
known  as  “Chatter  Program” 

-  CAE’s  Ground  and  Air  Traffic  Environment  System  (GATES) 

■  Personal  Computer  Commercial  Off-The-Shelf  Systems  (PC  COTS) 

■  Intelligent  Systems 

■  Voice  Generation  and  Recognition  Technologies 

Two  systems  specifically  designed  for  commercial  air  carrier  simulation  use  were  identified: 
IRAS  and  GATES.  IRAS  is  a  United  Airlines  in-house  development  with  very  high  operational 
realism.  It  is  based  on  field  recordings  of  actual  radio  communications  on  the  appropriate  routes 
and  includes  a  meaningful  partyline.  By  dubbing  ATC  with  the  respective  I/E  voice,  the  I/E  can 
intervene  without  the  pilot  trainee  realizing  it.  It  was  eventually  abandoned  due  to  technical 
difficulties  with  trigger  algorithms  and  the  high  cost  of  scenario  and  system  integration.  GATES 
is  a  simulator  add-on  offering  a  representation  of  visual  and  radio  communications  terminal 
traffic.  It  is  fully  integrated  with  aircraft  movement  and  other  simulation  events.  Realism  is 
increased,  according  to  anecdotal  feedback  from  some  users.  It  does  not,  however,  provide  any 
communications  to  or  from  own  airplane. 

Radio  communications  simulation  applications  at  the  lower  end  of  the  flight  simulator  market 
were  also  identified.  The  PC  COTS  flight  simulator  market  clearly  perceives  a  market  value  for 
including  realistic  radio  communications  add-ons.  Flexibility  and  controllability  vary  greatly 
among  products.  The  relevance  of  the  partyline  and  the  ability  to  send  messages  to  ATC  is 
limited.  Generally,  these  programs  are  limited  to  frequency  chatter  and  very  limited  ATC 
directives  to  the  simulated  aircraft. 

Enabling  technologies  that  would  support  the  advancement  of  realistic  radio  communications 
simulation  include  intelligent  systems,  and  voice  generation  and  recognition  technologies. 
Intelligent  systems  have  the  potential  for  a  high  degree  of  communication  realism  with  no 
instructor  intervention;  they  use  rule-based  models  of  goals  and  behavior  of  speaker  (e.g.. 
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controller).  Messages  are  generated  based  on  speaker  task  goals  and  situation  awareness,  thus 
eliminating  the  need  for  complex  “triggering”  algorithms  or  complex  scenario  scripting. 

Finally,  any  successful  radio  communications  simulation  system  would  have  to  rely  on  speech 
generation  and  recognition  systems.  Great  progress  has  been  made  in  both  fields.  Both  rule-based 
synthetic  speech  and  digitized  natural  concatenated  speech  achieve  intelligibility  ratings  rivaling 
natural  speech,  but  voice  quality  is  still  an  issue  especially  in  a  high-workload  environment.  The 
main  problem  for  voice  recognition  system  remains  speaker  variability;  even  with  a  highly 
regulated  phraseology  such  as  that  used  in  aviation,  extensive  “training”  of  the  system  is  required 
for  reliable  recognition  of  the  utterances  of  a  particular  speaker. 

CONCLUSIONS 

The  following  findings  are  documented  in  this  report: 

■  The  literature  emphasizes  the  need  for  a  realistic  training  and  evaluation  environment. 

■  Subject  matter  experts  perceive  realistic  radio  communications  as  important. 

■  Airlines  mostly  leave  the  simulation  of  radio  communications  to  the  already  very  busy 
individual  instructor/evaluators  due  to  cost  considerations. 

■  The  technology  to  simulate  ATC/Company  and  partyline  automatically  is  still  immature, 
although  there  are  some  promising  emerging  technologies. 

The  underlying  premise  of  this  work — that  the  provision  of  realistic  radio  communications 
during  training  and  evaluation  of  airline  pilots  would  enhance  safety — has  been  confirmed  thus 
far.  The  airlines,  however,  are  unlikely  to  commit  the  funds  required  for  the  provision  of  realistic 
radio  communications  in  the  absence  of  a  documented  gain  in  safety  and  an  FAA  requirement. 
The  technology  development  seems  to  be  largely  market  driven,  but  would  of  course  be  furthered 
by  such  a  requirement.  Therefore,  a  proof  of  concept  is  required  that  will  empirically  demonstrate 
the  need  for  realistic  radio  communications. 

A  first  step  may  be  to  systematically  compare  the  communication  load  and  its  effect  on  pilot 
workload  and  CRM  behaviors  as  a  function  of  radio  communications  during  actual  operations  in 
the  air  versus  simulator  Line  Oriented  Flight  Training/Line  Operational  Evaluations 
(LOFT/LOEs).  Another  step  may  be  an  assessment  of  pilots  who  have  just  completed  the  initial 
qualification  in  their  first  fleet.  The  question  of  interest  would  be  how  prepared  such  pilots  are  to 
handle  ATC  and  company  communications  requirements  during  their  first  Initial  Operating 
Experience  (IOE)  flights.  This  may  be  followed  up  by  a  carefully  conceived  simulator  study 
examining  the  benefits  of  different  levels  of  radio  communications  realism  for  different  training 
and  event  types.  Full  skill  transfer  to  and  from  the  airplane  is  a  critical  issue,  if  simulator  use  for 
training  and  evaluation  is  to  be  mandated. 
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1.  INTRODUCTION 


1.1  PROJECT  BACKGROUND  AND  MOTIVATION 

This  project  is  part  of  an  overall  effort  of  the  Federal  Aviation  Administration  (FAA)  to  ensure 
that  airplane  simulator  fidelity  is  commensurate  with  its  current  uses.  Earlier  simulators 
permitted  training  and  checking  of  procedures,  system  knowledge,  and  navigation,  but  did  not 
represent  the  airplane  faithfully  enough  to  permit  training  and  evaluation  of  flying  skills  and 
complex  decision  making.  In  fact,  pilots  often  explained  that  they  passed  their  in-flight  checks 
despite  previous  exposure  to  those  low-level  fidelity  simulators,  indicating  the  danger  of 
negative  training  in  an  unsuccessful  simulation  of  the  airplane  (Boothe,  2000). 

It  was  not  until  the  advent  of  the  FAA’s  Advanced  Simulation  Plan  in  1980,  authorizing  much 
more  extensive  airline  pilot  training  and  checking  credit  in  the  simulator,  that  technical  simulator 
requirements  were  specified  (Federal  Register,  1980,  CFR  FAR  Part  121,  Appendix  H).  Today, 
total  initial,  transition,  and  upgrade  training  and  checking  of  airline  pilots  may  occur  in  the 
simulator,  followed  by  initial  operating  experience  but  no  further  training  and  checking  in  the 
airplane.  This  has  virtually  eliminated  training  accidents  since  at  least  all  major  airlines  train  and 
check  dangerous  maneuvers  safely  in  the  simulator.  In  the  near  future,  it  is  expected  that  the 
FAA  will  make  simulator  use  mandatory  for  all  training  and  qualification. 

Current  simulator  qualification  criteria  ensure  that  flight  performance,  handling  qualities,  and 
simulator  models  are  suitable  for  training  and  checking  flying  skills  in  addition  to  procedural 
knowledge  [FAA  AC120-40A,  B,  and  C  (draft)].  With  both  the  growing  complexity  of  flight 
deck  automated  systems  and  the  concurrent  elimination  of  the  flight  engineer  position,  however, 
increased  emphasis  is  being  placed  on  training  and  evaluation  of  system  management  and  crew 
resource  management  (CRM)  skills.  This  is  accomplished  through  the  use  of  LOFT  (Line 
Oriented  Flight  Training)  and  LOE  (Line  Operational  Evaluation).  LOFT  and  LOE  both  require 
high  fidelity  flight  simulators  and  carefully  constructed  realistic  scenarios  designed  to  present  to 
crews  a  series  of  events  that  require  crew  coordination  and  system  management  skills.  FAA’s 
Advanced  Qualification  Program  (AQP),  an  alternative  operations  personnel  qualification 
program,  emphasizes  full  mission  simulations  containing  complex  scenarios  involving 
contingencies  that  may  contribute  to  accidents  and  incidents  (Federal  Register,  1990,  CFR  Part 
121  SFAR  58).  It  is  primarily  in  the  context  of  AQP  requirements  that  the  question  arises  as  to 
what  level  of  fidelity  the  operational  environment  must  be  simulated  to  ensure  full  transfer  of 
pilot  performance  and  workload,  from  the  simulator  to  the  airplane  for  training  and  vice  versa  for 
evaluation. 

For  this  purpose,  realistic  simulation  of  air  traffic  control  (ATC)  and  company  radio 
communications  appears  to  be  essential.  However,  the  simulation  of  such  communications  seems 
to  be  often  left  to  the  instructor/evaluator’s  (I/E)  role-play.  This  potentially  increases  I/E 
workload,  thus  reducing  their  ability  to  observe  the  crews  and  operate  the  simulator.  At  the  same 
time,  it  potentially  decreases  the  workload  of  pilots  compared  to  what  they  would  experience  in 
actual  flight  operations.  This  latter  effect  in  turn  may  reduce  the  realism  and  effectiveness  of 
both  pilot  training  and  evaluation.  Moreover,  I/Es  may  vary  in  the  way  they  simulate  radio 
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communications  for  a  particular  scenario,  leading  to  variations  in  the  complexity  of  the 
environment  that  crews  are  exposed  to.  This  may  result  in  uneven  training  and  unfair  evaluation. 
This  lack  of  standardization  was  pointed  out  by  an  I/E  who  remarked  that  the  provision  of  radio 
communications  is  “all  relative  to  the  creative  ability  of  the  instructor.” 

1.2  PROJECT  GOALS 

This  project  addresses  the  requirement  for  and  feasibility  of  providing  a  realistic  radio 
communications  environment  during  full  mission  simulations  in  an  at  least  partially  automated 
fashion.  Realistic  radio  communications  include  air  traffic  control  and  company  voice 
communications  to  own  airplane,  as  well  as  ATC  communications  to  and  from  other  aircraft 
controlled  by  the  same  controller  via  a  shared  communications  frequency  (i.e.,  partyline).  Such 
communications  would  have  to  be  appropriate  for  the  airspace  and  responsive  to  crew  actions. 
They  should  reflect  normally  occurring  ATC-driven  timing  that  cannot  take  pilots’  activities  or 
workload  into  account,  as  an  instructor/evaluator  who  is  aware  of  crew  activities  might  do.  They 
also  should  contain  a  meaningful  partyline  that  pilots  can  use  to  enhance  their  traffic  and  weather 
awareness.  Frequency  monitoring  is  important  for  pilots  waiting  for  an  opportunity  to  be  heard 
on  a  congested  frequency,  helping  to  avoid  interrupting  an  ongoing  exchange  by  keying  the 
microphone.  Other  communication  problems  common  in  the  air  that  may  merit  simulation 
include  pilots  that  are  unaware  that  their  activated  microphone  is  blocking  an  entire  frequency, 
non-standard  FAA  phraseology  in  airspace  using  International  Civil  Aviation  Organisation 
(ICAO)  phraseology,  or  dealing  with  non-native  English-speaking  controllers.  A  recurring 
problem  when  flying  in  international  airspace  is  the  use  of  non-FAA  units  such  as  kilometers 
instead  of  nautical  miles,  kilograms  instead  of  pounds  of  fuel,  pressure  in  millibars  instead  of 
mmHg,  or  the  transition  altitude  for  barometric  corrected  altitude  instead  of  standard  pressure 
(Flight  Levels).  But  even  when  simulating  FAA  controlled  airspace,  communication  problems 
such  as  complex  instructions  at  a  high  speech  rate,  non-standard  phraseology,  failure  to  correct 
pilots’  incorrect  responses  to  a  clearance,  callsign  confusions,  and  even  dealing  with  conceptual 
errors  by  controllers  should  be  practiced  (Biirki-Cohen,  1995;  Cardosi,  1993;  Rodgers  &  Nye, 
1993). 

The  underlying  premise  of  this  research  is  that  providing  a  realistic  radio  communications 
environment  could  have  a  significant  impact  on  safety  by  enhancing  both  pilot  training  and 
evaluation.  A  realistic  radio  communications  environment  would  provide  pilots  with 
opportunities  to  enhance  their  time-sharing  as  well  as  task  and  distraction  management  skills.  It 
would  also  improve  pilots  CRM  and  coordination  with  company  and  ATC.  The  latter  may  be 
especially  important  when  training  or  evaluating  new  procedures  such  as  landing  on  parallel 
runways  involving  an  additional  controller.  A  meaningful  partyline  containing  information  on 
traffic  and  weather  should  reinforce  careful  listening  to  improve  situational  awareness.  Realistic 
radio  communications  also  help  ensure  currency  in  ATC  communication  procedures  and 
phraseology,  as  well  as  attention  allocation  and  differentiation  skills  necessary  (e.g., 
distinguishing  clearances  for  one’s  own  airplane  from  clearances  for  other  aircraft).  Automated 
provision  of  radio  communications  should  free  instructors/evaluators  to  focus  more  of  their 
attention  on  instructing  and  observing  the  crew  without  the  competing  task  of  providing  radio 
communications.  Perhaps  most  important,  the  provision  of  realistic  radio  communications  may 
help  avoid  exposure  to  an  impoverished  environment  potentially  leading  to  the  development  of 
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tunnel  vision,  where  pilots  focus  on  aviating  skills  alone  while  losing  situational  awareness  and 
multitasking  skills.  The  overall  gain  in  realism  may  help  pilots  to  rid  themselves  of  the  so-called 
“simulator  mindset,”  which  could  interfere  with  transfer  of  pilot  performance  and  behavior  to 
and  from  the  airplane. 

This  report  documents  the  first-year  efforts  in  this  project,  which  focused  on  radio 
communications  without  considering  advanced  datalink  technologies.  Once  such  technologies 
are  implemented,  certain  aspects  of  this  work  regarding  the  means  of  information  exchange  with 
ATC  and  company  as  well  as  the  content  of  the  partyline  will  have  to  be  reconsidered.  For  the 
near  future,  however,  voice  communications  via  congested  approach  and  terminal  radio 
frequencies  are  likely  to  remain  the  dominant  form  of  air-ground  information  exchange.  First,  a 
multi-disciplinary  literature  review  was  conducted  to  determine  the  need  for  realistic  radio 
communications  from  a  theoretical  perspective.  The  main  findings  of  this  review  are  included  in 
the  body  of  the  report;  Appendix  A  provides  additional  information  on  the  role  of  ATC 
communications  in  accidents  and  incidents  and  Appendix  B  elaborates  on  both  the  need  and  the 
methods  of  task  management  training.  Second,  a  survey  of  I/Es  determined  whether  and  how 
airlines  are  currently  simulating  radio  communications.  The  same  survey  also  collected  I/Es 
opinions  on  the  impact  of  realistic  radio  communications  simulation.  Third,  a  review  of 
technologies  that  may  support  the  provision  of  realistic  radio  communications  was  conducted. 
The  results  and  conclusions  from  these  efforts  are  described  in  detail  in  the  following  sections. 
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2.  LITERATURE  REVIEW 


The  literature  review  focused  on  the  relevant  aspects  of  AQP  and  CRM  to  realistic  radio 
communications  simulation.  Although  communication  skills  are  critical  in  achieving  technical 
and  procedural  objectives  as  well  as  CRM  objectives  in  flight,  most  of  the  communication  skills 
literature  focuses  on  CRM  (Kanki  and  Smith,  in  press).  Therefore  the  following  section  reviews 
AQP  and  CRM  in  general,  CRM  between  the  flight  crew  and  ATC,  and  “task  management”  as  a 
specific  example  of  how  realistic  radio  communications  simulation  can  enhance  CRM  training. 

AQP  and  CRM  are  closely  connected.  One  of  the  goals  of  AQP  is  to  integrate  training  and 
evaluation  of  CRM  skills  at  each  stage  of  the  curriculum.  Qualification  and  continuing 
qualification  curricula  are  required  to  include  an  LOE,  consisting  of  a  full  flight  scenario 
systematically  designed  to  target  specific  technical  and  CRM  skills.  Demonstration  of 
proficiency  in  maneuver-oriented  technical  skills  is  a  “necessary  but  insufficient  condition  for 
pilot  qualification”  (Longridge,  1997,  emphasis  added).  CRM  training  and  evaluation  addresses 
human  factors  such  as  leadership,  communication  skills,  time  management,  situational 
awareness,  and  attitudes  in  flight  operations  (FAA,  1990,  AC120-35B,  Crew  Line  Operational 
Simulation,  LOS).  It  is  an  integral  part  of  any  AQP,  and  both  the  AQP  and  CRM  literature 
underscore  the  importance  of  realistic  scenarios. 

2.1  ADVANCED  QUALIFICATION  PROGRAM 

The  founding  principle  of  AQP  is  that  training  and  evaluation  of  pilots  should  be  based  on  the 
activities  encountered  on  the  operational  job.  A  first  step  in  AQP  is  thus  an  analysis  of  all  tasks 
that  a  pilot  needs  to  perform  during  actual  operations,  which  then  guides  curriculum  and  scenario 
development  in  the  simulator.  Each  task  is  given  a  label  describing  what  type  of  skill  is  required 
to  perform  it.  The  AQP  task-listing  example  found  on  the  FAA  AQP  Management  Web  site 
(http://www.faa.gov/avr/afs/agphone.htm).  clearly  shows  that  coordination  with  company  and 
ATC  is  an  integral  part  of  line  operations  and  that  frequency  monitoring  is  important  for 
maintaining  traffic  and  weather  situational  awareness.  Most  activities  related  to  radio 
communications  are  coded  as  “attitudes.”  Those  that  are  part  of  situation  assessment  are  labeled 
“cognitive  skills,”  and  tuning  the  radio  is  considered  a  “psychomotor  skill.” 

Communications  with  dispatch  and  other  “appropriate  agencies  (i.e.,  weather,  AFSS  [Automated 
Flight  Service  Station],  maintenance,  company),”  in-person  or  via  telephone,  start  during  pre¬ 
departure  ground  mission  preparation.  After  all  the  radios  have  been  prepared  for  flight,  the  final 
cockpit  preparations  include  communications  with  company  and  maintenance  and  obtaining 
departure  ATIS  (Automatic  Terminal  Information  Service)  and  ATC  clearance  followed  by 
Pushback/Start  Clearance  via  radio/ACARS  (Aircraft  Communications  Addressing  and 
Reporting  System).  The  pushback  procedure  includes  interphone  communication  with  the  ground 
crew,  followed  by  the  taxi  clearance  procedure  via  ACARS/radio.  Finally,  ATC  take-off 
clearance  is  obtained.  This  completes  routine  communications  tasks  on  the  ground,  unless  the 
“assessment  of  the  take-off  environment”  leads  to  an  adjustment  to  the  original  ATC  instructions. 
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While  the  requirement  for  assessing  the  take-off  environment  (weather  and  airfield  conditions)  is 
explicitly  spelled  out  as  part  of  the  take-off  performance,  there  is  no  mention  of  frequency  or 
partyline  monitoring  as  a  means  of  obtaining  information  for  situation  awareness  in  the  terminal 
environment.  Situation  awareness  has  been  described  as  the  pilots  knowledge  of  the  environment 
at  a  given  point  in  time,  and  more  specifically,  of  which  subsets  of  the  environment  are 
important  for  the  task  at  hand  (Endsley,  Farley,  Jones,  Midkiff,  and  Hansman,  1998).  At  any 
stage  of  operations,  pilots  need  to  be  aware  of  the  surrounding  traffic,  terrain,  weather,  and  status 
of  aircraft  systems.  To  arrive  at  the  desired  destination,  they  also  need  to  be  aware  of  their 
position.  With  the  exception  of  system  awareness,  ATC  clearances  to  own  and  other  aircraft  play 
a  critical  role  in  maintaining  situational  awareness.  In  the  AQP  task  list,  frequency  monitoring  is 
first  alluded  to  in  the  section  “Perform  Enroute  Situational  Assessment”  for  the  acquisition  of 
weather  awareness  (“Monitor  weather  using  cues  from... radio  communications... and  ACARS 
reports.”)  In  subsequent  phases,  reference  is  made  to  cues  from  ATC  both  for  weather  and 
maintaining  traffic  separation. 

Throughout  the  next  phases,  routine  radio  communications  (or  declaration  of  emergencies)  with 
the  appropriate  controlling  ATC  agencies  continue  to  be  listed,  including  tuning  the  radio  to  the 
appropriate  frequency.  En-route  communications  procedures  warrant  their  own  section, 
including  subsections  describing  cockpit  and  cabin  crew  as  well  as  company  and  ATC 
communications.  Of  these,  cockpit  crew  communications  take  the  lead  with  16  items  listed, 
followed  by  five  company  communications  items  (including  ACARS).  The  cabin  crew  and  ATC 
communications  subsections  contain  three  items  each.  During  subsequent  phases,  cockpit  crew 
communications  continue  to  lead,  followed  by  ATC  and  company  (radio/ACARS).  In  short, 
coordination  with  company  and  ATC  as  well  as  maintaining  situational  awareness  form  an 
integral  part  of  the  AQP  task  list  (FAA  2000),  and  messages  from  ATC  are  considered  an 
essential  ingredient  to  ensuring  the  realism  of  full  mission  simulations  (FAA,  1990,  AC120-35B, 
Crew  Line  Operational  Simulation,  LOS). 

2.2  CREW  RESOURCE  MANAGEMENT 

The  importance  of  CRM  started  to  be  recognized  in  the  late  1970’s,  when  the  human  error 
aspects  of  airplane  crashes  were  identified  as  failures  of  interpersonal  communications,  decision 
making  and  other  leadership  and  crew  coordination  skills  (Helmreich  and  Foushee,  1993; 
Kayten,  1993).  Originally  standing  for  “Cockpit  Resource  Management,”  early  CRM  programs 
focused  on  cockpit  group  dynamics.  As  some  airlines  started  to  extend  their  CRM  efforts  to 
other  groups  involved  in  airplane  operations,  the  phrase  “Crew  Resource  Management”  was 
adopted. 

The  Advisory  Circular  on  Crew  Resource  Management  Training  explicitly  lists  not  only  onboard 
cabin  personnel,  but  also  ground-based  maintenance  personnel,  aircraft  dispatchers,  and  air 
traffic  controllers  as  part  of  the  CRM  process  (FAA,  1998,  AC  120-5 1C,  Crew  Resource 
Management  Training,  CRM).  In  a  section  on  extending  training  beyond  the  cockpit,  the 
advisory  circular  highlights  the  benefits  of  using  real  air  traffic  controllers,  aircraft  dispatchers, 
and  maintenance  personnel  during  LOFT  sessions.  A  1996  Air  Line  Pilot  article  on  flight  deck 
confusion  concluded,  from  a  survey  of  recent  Aviation  Safety  Reporting  System  (ASRS)  reports 
explicitly  mentioning  pilot  confusion,  that  better  communications  between  cockpit  crews  and 


6 


ATC  controllers  could  help  prevent  confusion-related  incidents  (Rosenthal,  Chamberlin,  and 
Matchette,  1996). 

Communication  deficiencies  included  lack  of  compliance  with  standard  phraseology  and 
communication  protocols.  Thirty  of  the  100  reports  involved  amended  clearances,  and  frequently 
pilots  refrained  from  asking  controllers  for  clarification  of  a  confusing  clearance.  Rosenthal  et  al. 
state  that  pilot  confusion  can  “best  be  prevented  through  continuing  emphasis  on  crew 
performance,  with  the  understanding  that  ATC  is  a  key  member  of  the  flight  team,”  and 
recommend  that  CRM  place  greater  emphasis  on  crew/ ATC  interactions.  There  are  many  other 
ASRS  reports  (see  Billings  &  Cheaney,  1981;  Connell,  1994,  1996),  research  investigations  (see 
Kanki  and  Palmer,  1993;  Predmore,  1991),  and  industry,  military  and  general  aviation  magazine 
articles  that  discuss  the  role  of  communication  in  incidents  and  accidents.  Numerous  industry 
and  government  working  groups  have  also  focused  reviews  on  operational  communications  and 
their  ongoing  problems  in  the  system. 

2.2.1  Approach  and  Landing  Accident  (ALA)  Reduction  Task  Force  Report 

A  recent  review  of  approach  and  landing  accidents  by  the  Flight  Safety  Foundation  again  pointed 
to  the  importance  of  efficient  interactions  with  ATC  and  company  (Khatwa  &  Helmreich,  1999). 
The  task  force  based  its  conclusions  on  an  analysis  of  76  ALAs  and  serious  incidents  involving 
jet  and  turboprop  aircraft  and  occurring  between  1984  and  1997  (inclusive).  “Incorrect  or 
inadequate  ATC  instruction/advice/service”  was  a  causal  factor  in  33  percent  of  all  occurrences. 
It  ranked  eleventh  among  the  most  common  causal  factors  after  “Poor  professional 
judgment/airmanship”  in  first,  “Lack  of  positional  awareness”  in  fourth,  and  long  before 
“Interaction  with  automation”  in  seventeenth  place.  “[DJemanding  ATC  clearances”  are  also 
explicitly  mentioned  in  context  with  even  higher  ranking  causal  factors  such  as  the  sixth-placed 
“Flight-handling  difficulties,”  the  eighth-placed  “Press-on-itis,”  or  the  tenth-placed  “Slow  and/or 
low  on  approach.”  In  many  cases  of  “press-on-itis,”  “a  breakdown  in  CRM  between  the  flight 
crew  and  ATC”  was  observed.  Further  description  of  results  of  this  and  other  studies  supporting 
its  findings,  including  a  discussion  of  problems  involving  non-native  English  speaking  ATC  or 
crews,  can  be  found  in  Appendix  A. 

2.2.2  Realistic  Radio  Communications  for  Task  Management  Training 

Chou  et  al.  have  investigated  the  significance  and  nature  of  the  task  management  process  in 
flight  operations  by  first  conducting  a  review  of  National  Transportation  Safety  Board  (NTSB) 
accident  and  ASRS  incident  reports  and  then  testing  their  findings  in  a  controlled  simulator 
experiment  (Chou,  Madhavan,  and  Funk,  1996).  They  describe  cockpit  task  management  (CTM) 
as  consisting  of  the  following  seven  functions:  1)  task  initiation  when  appropriate  conditions 
exist;  2)  task  monitoring;  3)  task  prioritization;  4)  resource  allocation;  5)  task  interruption  when 
resources  need  to  be  allocated  to  a  higher  priority  task;  6)  task  resumption  when  priorities 
change  or  resources  become  available;  and  7)  task  termination  when  task  has  been  completed, 
cannot  be  completed,  or  has  become  irrelevant. 

From  this  functional  description,  it  is  clear  that  many  of  these  functions  implicitly  refer  to  intra¬ 
cockpit  communications  that  serve  to  achieve  crew  coordination,  resource  management  and  task 
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prioritization.  However,  it  is  also  easy  to  see  how  these  functions  depend  upon  direct  information 
transfer  between  cockpit  crew,  ATC  and  company,  and  indirect  information  gained  from 
frequency  monitoring. 

However,  there  is  substantial  evidence  from  accident  and  incident  data  as  well  as  research  that 
the  capacity  to  capture  and  use  information  is  limited.  Furthermore,  this  limitation  can  lead  to 
human  error.  For  instance,  radio  communications  were  found  to  be  one  of  the  primary 
contributors  to  monitoring  errors  (Sumwalt,  Morrison,  Watson,  and  Taube,  1997).  In  this  study, 
200  ASRS  reports  (submitted  between  1992  and  1996)  were  reviewed  for  monitoring  errors 
which  were  defined  as  a  “failure  to  adequately  watch,  observe,  keep  track  of,  or  cross  check”  the 
aircraft’s  trajectory  or  systems.  Consequences  encompassed  a  wide  range  of  safety  problems, 
including  taxi,  flight  path,  or  speed  deviations,  runway  incursions,  system  damage,  and  loss  of 
control.  For  the  170  reports  that  indicated  the  tasks  or  functions  accomplished  shortly  before  or 
during  the  error,  42  percent  of  the  reports  involved  radio  communications  (ATC,  company,  or 
obtaining  ATIS).  (See  fuller  description  of  this  and  other  studies  in  Appendix  B.) 

In  order  to  train  and  evaluate  pilots  on  task  management  skills,  significant  information  resources 
are  omitted  when  radio  communications  are  not  simulated  in  a  realistic  manner.  A  similar  but 
distinct  issue  is  the  role  of  radio  communications  as  a  task  distractor.  That  is,  pilots  should  be 
trained  to  maximize  their  use  of  radio  communications  as  an  information  source  (especially 
under  non-normal  operational  or  procedural  conditions),  but  they  also  should  be  trained  on 
interruption/distraction  management  skills  within  a  realistic  ATC  training  environment  (see 
Latorella,  1996).  Also  see  Appendix  B  for  a  continued  discussion  of  Task  Management  Training 
Methods,  including  a  discussion  of  whole-task  versus  part-task  training. 

2.3  SUMMARY 

As  an  example  of  one  type  of  CRM  skill,  the  survey  of  the  task  management  literature  shows 
how  important  it  is  for  pilots  to  be  exposed  to  all  aspects  involved  in  flying  an  airplane.  One 
reason  why  whole-task  training  in  a  fully  loaded  environment  including  radio  communications 
may  be  superior  to  part-task  training  in  an  incomplete  environment  is  that  it  may  avoid  the  false 
sense  of  operational  simplicity  that  comes  with  tunnel  vision.  Tunnel  vision  implies  that  pilots 
are  trained  to  concentrate  on  certain  aspects  of  the  flying  task  while  learning  to  ignore  others.  To 
avoid  tunnel  vision,  a  complete  mental  model  of  the  task  needs  to  be  acquired  during  training.  As 
Mangold  and  Eldredge  (1993)  put  it,  the  mental  model  serves  as  a  structure  for  understanding  the 
task  and  developing  expectations  as  to  the  events  that  will  occur. 

Bransford  and  Franks  (1976)  subscribe  to  a  stage-setting  metaphor  of  learning,  where  past 
experiences  prepare  the  ground  for  dealing  with  novel  situations.  Learning  is  an  active  process, 
and  practice  can  lead  to  either  activation  or  inhibition  of  cognitive  pathways.  If  pilots  are 
consistently  exposed  to  an  impoverished  environment  during  training  compared  to  the  real 
world,  they  may  end  up  unprepared  for  the  complexity  of  the  flying  task  in  the  air.  This  was  also 
confirmed  by  one  of  the  participants  in  the  I/E  survey,  who  wrote,  “without  communication 
simulation,  when  the  pilot  trainee  finally  arrives  in  the  ‘real  world,’  he  must  add  another 
component....  This  new  component  can  really  complicate  line  flying.” 
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3.  INSTRUCTOR/EVALUATOR  SURVEY 


A  questionnaire  on  realistic  radio  communications  practices  and  opinions  was  presented  to  the 
AQP  LOFT  Instructor/Evaluator  Focus  Group  in  May  1999.  A  few  additional  questionnaires 
were  sent  to  airlines  over  the  succeeding  months.  A  total  of  36  responses  were  received.  Six  of 
these  reported  on  training  events  for  non-scheduled  aircraft  that  are  not  representative  of  Part 
121  operations  or  AQP.  One  additional  questionnaire  was  excluded  because  the  respondent 
indicated  that  he  did  not  understand  the  distinction  between  clearances  to  own  airplane  and 
partyline.  The  29  responses  included  in  the  final  analysis  stemmed  from  seven  major,  one  cargo, 
four  regional,  and  two  foreign  (AQP)  airlines.  Respondents  averaged  some  10  years  of  pilot 
instruction  and  evaluation. 

3.1  CURRENT  PRACTICES  OF  RADIO  COMMUNICATIONS  SIMULATION 

Instructor/Evaluators  were  queried  on  their  company’s  current  practices  of  simulating  air  traffic 
control  communications  and  company  communications  to  own  and,  for  ATC,  other  aircraft  (the 
so-called  partyline).  They  were  also  queried  regarding  their  company’s  visual  and  Traffic  Alert 
and  Collision  Avoidance  System  (TCAS)  simulation  of  the  traffic  environment. 

3.1.1  Radio  Communications  Simulations  to  Own  Airplane 

The  first  question  asked  I/Es  to  indicate  with  a  checkmark,  which  types  of  communications  to 
own  airplane  their  airline  simulated  during  different  training  and  evaluation  events  (Appendix  C, 
Question  1).  The  question  was  subdivided  into  ATC  and  company  communications.  The  second 
question,  again  subdivided  into  ATC  and  company  communications,  asked  I/Es  to  indicate  how 
these  communications  were  implemented. 

Communications  Simulated  During  LOFT 

Twenty-seven  answers  on  communications  during  LOFT  were  gathered  from  the  29  respondents 
(see  Table  1).  One  I/E  did  not  answer  this  question,1  and  another  indicated  that  he  is  only 
familiar  with  LOEs  and  SPOT  (Special  Purpose  Operational  Training). 

With  regard  to  ATC  communications,  all  of  the  LOFT  respondents  indicated  simulating 
communications  from  the  tower  ground,  tower  local,  and  approach/departure  controller.  Twenty- 
four  (89  percent)  of  respondents  reported  simulating  Automatic  Terminal  Information  Service 
(ATIS).  Twenty-five  respondents  (close  to  93  percent)  reported  simulating  tower  clearance 
delivery  and  air  route  traffic  control  center  (ARTCC)  communications.2 


1  This  I/E  did,  however,  later  answer  questions  regarding  the  effect  of  his  company’s  radio  communications 
practices  on  workload.  His  data  was  included  assuming  the  same  practices  as  indicated  by  his  three  colleagues  from 
the  same  airline. 

2  One  of  the  two  I/Es  that  indicated  not  simulating  ARTCC  communications  does  so  during  LOEs.  The  other 
indicated  participating  only  in  LOFT  and  has  two  colleagues  from  the  same  airline  who  did  indicate  simulating 
ARTCC. 
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Only  one  of  the  27  LOFT  participants  indicated  that  he  did  not  simulate  any  company 
communications  (see  Table  2).  All  but  one  of  the  remaining  26  respondents  (93  percent  of  all 
LOFT  participants)  reported  simulating  communications  with  dispatchers.  Nineteen  (70  percent) 
indicated  simulating  communications  with  ramp  or  gate  personnel.  Eighteen  respondents  (67 
percent)  indicated  simulating  cabin  personnel  communications.  In  the  field  “Other”  18  I/Es  (67 
percent)  entered  “maintenance,”  one  “ground  crew,”  one  “passenger  service  ”  and  another 
“emergency  vehicles”  (4  percent  each). 

Table  1.  Types  of  ATC  Communications  Simulated  in  LOFT,  LOE,  and  SPOT 


Environment 

LOFT 

Total  n  =  27 

LOE 

Total  n  =  19 

SPOT 

Total  n  =  18 

Responses 

Percent 

Responses 

Percent 

Responses 

Percent 

ATIS 

24 

89 

18 

95 

13 

72 

Clearance 

25 

93 

17 

89 

13 

72 

M  Tower  Ground 

27 

100 

19 

100 

17 

94 

Tower  Local 

27 

100 

19 

100 

18 

100 

/  Approach/Dep. 

27 

100 

19 

100 

17 

94 

Center 

25 

93 

19 

100 

16 

89 

Table  2.  Types  of  Company  Communications  Simulated  in  LOFT,  LOE,  and  SPOT 


Environment 

LOFT 

Total  n  =  27 

LOE 

Total  n  =  19 

SPOT 

Total  n  =  18 

Responses 

Percent 

Responses 

Percent 

Responses 

Percent 

Dispatch 

25 

93 

18 

95 

14 

78 

Ramp/Gate 

19 

70 

12 

63 

10 

56 

Cabin 

18 

67 

13 

68 

10 

56 

Maintenance 

18 

67 

16 

84 

11 

61 

Communications  Simulated  During  LOEs 

Nineteen  responses  were  available  regarding  communications  during  LOEs  (also  shown  in  Table 
1).  In  addition  to  the  one  skipped  question,  nine  respondents  apparently  do  not  perform  LOEs. 

With  respect  to  ATC,  all  participants  in  LOE  indicated  simulating  some  ATC  communications. 
All  respondents  checked  simulating  tower  ground,  tower  local,  approach/departure  and  ARTCC 
communications.  Eighteen  respondents  (95  percent  of  all  LOE  participants)  reported  simulating 
ATIS.  Seventeen  (89  percent)  reported  simulating  clearance  delivery  information. 


3  From  his  answer  to  a  question  later  in  the  survey,  it  can  be  concluded  that  the  I/E  that  skipped  the  question  also 
does  not  perform  LOEs. 
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Regarding  company  communications  (Table  2),  one  of  the  19  respondents  reported  simulating  no 
company  communications.  Eighteen  respondents  (95  percent)  reported  simulating  dispatch. 
Thirteen  respondents  (68  percent)  indicated  simulating  cabin  personnel  communications.  Twelve 
(63  percent)  I/Es  checked  the  ramp/gate  communications  box.  In  the  “Other”  box,  16 
respondents  (84  percent)  wrote  in  “maintenance,”  one  “ground  crew,”  and  another  “emergency 
vehicles”  (5  percent  each). 

Communications  Simulated  During  SPOT 

Eighteen  I/E  responses  were  available  regarding  communications  during  SPOT  (see  Table  1).  In 
addition  to  the  one  I/E  that  omitted  his  answer,  10  respondents  indicated  that  they  did  not 
participate  in  SPOT.4 

All  I/Es  participating  in  SPOT  simulate  tower  local  communications.  Seventeen  respondents  (94 
percent)  simulate  tower  ground  and  approach/departure  communications.  Sixteen  (89  percent) 
simulate  center  communications.  Thirteen  (72  percent)  simulate  ATIS  and  clearance  delivery. 

Company  communications  during  SPOT  (see  Table  2)  are  simulated  by  15  (83  percent)  of  the 
respondents.  Fourteen  (78  percent)  respondents  checked  the  company  dispatch  box.  Ten  (56 
percent)  indicated  simulating  ramp/gate  as  well  as  cabin  communications.  Eleven  respondents 
(61  percent)  wrote  in  “maintenance”  and  one  “ground  crew”  in  the  space  provided  in  the  “Other” 
box. 

Methods  of  Simulating  Radio  Communications  to  Own  Airplane. 

Alternative  methods  (shown  in  Appendix  C,  Question  2)  included: 

■  “I/E  provides”  all  radio  communications 

■  “Live  ‘controllers’  ”  (i.e.,  “an  additional  staff  member  focuses  on  radio  communications” 

■  “Recorded  controller  voice,” 

■  “Synthetic  controller  voice,” 

■  “Other” 

One  I/E  skipped  the  question  on  communication  simulation  methods  to  own  airplane,  leaving  28 
respondents  for  this  question. 

All  but  one  I/E  indicated  that  they  themselves  provided  all  company  communications  via  role- 
play.  The  one  exception  checked  “Other”  for  dispatch  and  indicated  in  the  comment  section  that 
many  communications  between  company  and  the  crews  were  provided  by  ACARS.  He  was  the 
only  I/E  representing  his  major  airline. 

With  respect  to  ATC  communications,  the  vast  majority  appears  to  be  provided  via  FE  role-play 
as  well.  Respondents  indicated  that  this  was  the  case  for  all  tower  ground,  tower  local, 
approach/departure,  and  ARTCC  communications  provided. 


4  As  is  evident  from  his  answer  to  a  later  question,  the  I/E  omitting  his  answer  to  this  question  also  does  not 
participate  in  SPOT. 
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Tower  clearance  delivery  and  ATIS  were  the  only  type  of  communications  to  own  airplane 
where  the  I/Es  did  not  always  indicate  having  to  role-play  communications.  Five  I/Es  (19  percent 
of  all  I/Es  indicating  their  method  of  clearance  delivery)  indicated  provision  of  clearance 
delivery  in  text-based  format  (printed  handouts,  ACARS).  A  majority  of  23  (82  percent) 
however  still  had  to  provide  this  information  themselves  (presumably  via  oral  communication). 

Twenty-seven  EEs  responded  to  the  question  of  ATIS  implementation.  Twelve  (44  percent) 
indicated  that  they  had  to  provide  this  information  themselves.  Fifty-six  percent  indicated  some 
supporting  system  which  may  depend  on  the  capabilities  of  a  particular  simulator.  Four  pilots 
checked  the  “EE  provides”  as  well  as  the  “Synthetic  controller  voice”  or  the  “Recorded 
controller  voice”  box.  Six  indicated  synthetic  and  two  recorded  natural  controller  voice.  Three 
wrote  in  “printer  paper”  or  “handouts”  in  the  space  provided  for  “Other.” 

Three  of  the  seven  respondents  from  one  airline  indicated  that  two  instructors  may  participate  in 
a  training  session,  at  least  when  instructing/evaluating  a  three-person  crew.5  This  gives  them  the 
opportunity  to  share  communication  duties.  One  EE  specified  that  the  instructor  for  the  pilots 
(Captain  and  First  Officer)  provides  the  ATC  radio  communications,  whereas  the  instructor  for 
the  flight  engineer  provides  the  company  communications. 

3.1.2  Simulation  of  Other  Aircraft 

The  next  set  of  questions  (Appendix  C,  Questions  4  and  5)  addressed  the  simulation  of  other 
traffic.  Dependent  on  phase  of  flight,  this  may  include  aircraft  and  ground  vehicles.  Other  traffic 
may  be  simulated  both  in  the  visual  and  the  auditory  modality.  Visual  simulation  may  include 
representations  of  other  traffic  in  the  out-the-window  view  and  on  the  TCAS.  Auditory 
simulation  of  other  traffic  consists  of  hearing  communications  from  and  to  other  aircraft  on  the 
same  air  traffic  control  radio  frequency,  the  so-called  partyline. 

The  first  question  (Appendix  C,  Question  4)  provided  boxes  to  be  checked  to  indicate  the  type  of 
vehicle  simulated  (aircraft  or  ground  vehicle)  in  each  modality  (out-the-window  view,  TCAS, 
communications  to  and  from  other  aircraft  or  ground  vehicles)  and  for  each  environment  (airport 
surface,  terminal,  or  en-route).  The  next  question  (Appendix  C,  Question  5)  provided  check 
boxes  regarding  the  implementation  of  communications  with  Clearance  Delivery,  Tower 
Ground,  Tower  Local,  and  Approach/Departure  control.  The  proposed  implementations  were 
“Not  simulated,”  “EE  provides,”  “Live  ‘controllers’,”  “Recorded”  and  “Synthetic  controller 
voice,”  and  “Other.” 

Five  EEs  left  both  questions  about  party  line  completely  blank.  All  these  (17  percent  of  all 
participants)  explicitly  commented  that  no  visual  simulation  of  other  traffic,  either  out-the- 
window  or  via  TCAS,  was  provided.  So  they  did  not  provide  any  communications  simulation 
either. 


5  All  but  one  of  the  remaining  four  pilots  from  this  same  company  reportedly  instruct  three-person  crews.  One  of 
these  started  to  add  a  similar  comment  into  the  space  provided,  but  crossed  it  out. 
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Table  3.  Simulation  of  ATC  Communications  with  Other  Aircraft  or  Vehicles  (Partyline) 


l§3j§  fjl  1 

Out-the-Window 

View 

Total  n  =  17 

TCAS 

Total  n  = 

10 

Communications 
to  Other 

Total  n  =  10 

Communications 
from  Other 

Total  n  =  9 

■piyiaBiii 

A/C 

14 

A/C 

8 

E/Vehicle 

8 

A/C 

2 

E/Vehicle 

4 

■■i 

A/C 

10 

A/C 

8 

A/C 

2 

A/C 

2 

A/C 

10 

5 

A/C 

2 

A/C 

2 

Note:  21  of  29  respondents  indicated  responses 

in  some  part  of  this  table. 

Table  3  gives  results  for  type  of  vehicle  simulated  in  each  modality  and  environment  (Appendix 
C,  Question  4).  Seventeen  (59  percent)  of  the  overall  I/Es  reported  some  out-the-window  view 
simulation  of  traffic.  On  the  airport  surface,  14  checked  simulation  of  aircraft  (A/C),  and  seven 
of  these  also  reported  simulating  emergency  vehicles  (E/ Vehicle).  One  additional  I/E  reported 
out-the-window  simulation  of  only  emergency  vehicles  on  the  surface.  Eight  reported  simulating 
out-the-window  aircraft  in  both  the  terminal  and  en-route  traffic  environment.  Two  of  these 
(from  the  same  airline)  did  not  indicate  any  visual  simulation  of  airport  surface  traffic.  Two 
indicated  simulating  terminal  but  not  en-route  out-the-window  aircraft,  and  two  en  route  but  not 
terminal.  The  latter  four  had  indicated  simulating  airport  surface  traffic  in  the  out-the-window 
view.  Only  eight  of  the  17  I/Es  reporting  out-the-window  simulation  of  traffic  also  reported 
simulating  communications  to  or  from  other  aircraft. 

Ten  I/Es  (34  percent)  reported  that  the  simulators  they  use  were  TCAS  equipped,  but  two  of 
these  indicated  that  it  was  available  only  on  the  airport  surface.  All  but  one  of  these  report  some 
out-the-window  view  simulation. 

Eleven  (38  percent)  of  all  I/Es  reported  providing  some  communications  to  or  from  other  aircraft 
or  vehicles,  at  least  on  the  surface.  Eight  of  these  provide  communications  both  ways,  two  only 
to  and  one  only  from  other  traffic.  On  the  surface,  eight  (28  percent)  reported  providing 
clearances  to  aircraft.  Two  of  these  reported  providing  instructions  also  to  ground  vehicles  (as  do 
two  additional  ones  that  do  not  provide  any  clearances  to  aircraft).  Of  the  I/Es  providing 
clearances  to  other  aircraft,  five  reported  that  they  also  simulate  pilot  readbacks.  One  additional 
respondent  indicated  that  he  provides  communications  only  from  aircraft  and  vehicles  on  the 
airport  surface.  All  the  I/Es  simulating  instructions  to  ground  vehicles  report  simulating 
responses  from  the  drivers.  Only  two  respondents  indicated  that  in  addition  to  simulating  surface 
communications,  they  simulate  clearances  and  readbacks  in  the  terminal  and  en  route 
environment.  One  I/E  explained  that  they  seldom  provide  terminal  and  en  route  communications 
due  to  I/E  workload. 
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Eighteen  I/Es  indicated  a  method  for  providing  communications  to  other  aircraft  (Appendix  C, 
Question  5).6  Most  indicated  that  they  simulated  communications  via  role-play,  with  few 
exceptions.  One  respondent  checked  the  “Other”  category  and  commented  that  two  instructors 
alternate  communications.  Another  checked  “Recorded  controller  voice”  for  tower  ground  and 
tower  local  and  Approach/Departure  control  communications.  For  tower  ground  and  local,  one 
last  respondent  checked,  in  addition  to  the  “I/E  provides”  box,  the  “Synthetic  controller  voice” 
box.  Follow-up  discussions  with  this  instructor  revealed  that  he  was  referring  to  the  availability 
of  GATES  (Ground  and  Air  Traffic  Environment)  on  some  simulators,  a  new  technology  that 
will  be  discussed  in  the  Technology  Review. 

3.1.3  Instructor/Evaluator  Workload  in  the  Simulator 

Instructor/Evaluators  were  also  asked,  for  each  simulation  event  type,  to  indicate  the  “percentage 
of  time  and  effort”  spent  “Running  (the)  Simulation,”  “Simulating  Radio  Communications,” 
“Instructing,”  “Observing,”  and  “Other”  with  the  indication  that  the  numbers  should  add  up  to 
100  percent  (Appendix  C,  Question  8a). 

All  29  I/Es  tried  to  answer  this  question.  However,  several  I/E  answers  were  omitted  due  to 
inconsistencies  in  their  responses.  In  addition,  I/Es  omitted  answers  for  some  but  not  all  training 
venues,  LOFT,  LOE  or  SPOT.  This  resulted  in  24  answers  for  LOFT,  16  for  LOE,  and  15  for 
SPOT  for  final  analysis. 

Only  three  I/Es  entered  any  information  into  the  “Other”  category.  One  indicated  that  for  LOFT 
and  LOE,  he  spent  1  percent  of  his  time/effort  providing  “cabin,  company  communications.” 
Two  indicated  that  they  spent  some  time  “[tjaking  notes.”  One  of  these  answered  only  for  LOE 
and  SPOT  and  indicated  spending  20  percent  for  both  events,  the  other,  who  answered  only  for 
LOFT,  indicated  spending  5  percent  of  time/effort  taking  notes.  The  20  percent  notetaker  was  the 
only  representative  of  his  airline.  So  as  not  to  distort  the  final  picture  on  the  basis  of  these  three 
respondents,  the  category  “Other”  was  eliminated  by  distributing  the  percentage  of  time/effort 
found  in  it  proportionally  over  the  four  other  categories  (running  simulation,  radio 
communications,  instructing,  and  observing)  shown  in  Figure  1 . 

Figure  1  shows  perceived  time/effort  allocation  averaged  across  EEs  for  each  training  event.  As 
can  be  seen,  during  all  three  events,  EEs  spend  most  of  their  time/effort  observing,  especially 
during  LOE  and  LOFT  (52  and  51  percent,  respectively).  Not  surprisingly,  this  percentage  drops 
down  to  37  percent  for  SPOT,  which  involves  “special  purpose”  training  and  shows  the  highest 
percentage  for  instructing,  namely,  25  percent  versus  8  percent  for  LOFT.  Of  course,  this 
percentage  is  lowest  for  LOE  with  4  percent.  Running  the  simulator  takes  the  same  amount  of 
time/effort  in  all  three  events,  i.e.,  22  percent.  Radio  communications  simulation  takes  slightly 
more  time  and  effort  during  evaluation,  22  percent  versus  20  percent  in  LOFT.  During  SPOT, 
radio  communications  appear  to  be  less  important,  with  the  percentage  of  time/effort  spent  on 
them  falling  to  16  percent. 


6  Only  nine  of  these  had  previously  indicated  that  they  provided  partyline  communications  in  Question  4. 
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Figure  1.  Perceived  I/E  Workload  in  the  Simulator 


These  percentages  are  important  because  they  depict  how  I/Es  provide  radio  communications 
while  performing  significant  other  tasks.  One  possible  exception  may  be  the  three  I/Es  from  one 
airline  instructing/evaluating  a  three-person  cockpit  and  with  support  from  a  second  I/E  for  the 
flight  engineer.  Although  these  reported  slightly  lower  percentages  for  radio  communications  in 
LOFT  (18  instead  of  20  percent),7  the  numbers  are  too  small  and  possibly  offset  by  the  extra 
effort  required  for  instructing  a  three-person  crew  to  allow  any  conclusions. 

3.2  SURVEY  OF  INSTRUCTOR/EVALUATOR  OPINION 

Instructor/evaluators  were  polled  on  their  opinions  on  the  effect  of  having  to  simulate  radio 
communications  on  workload  and  training  effectiveness,  as  well  as  on  the  importance  of 
simulating  the  partyline.  They  were  also  asked  for  their  opinions  on  the  overall  importance  of 
realistically  simulating  radio  communications  in  scenarios. 

3.2.1  Radio  Communications  and  Workload 

Instructors/evaluators  were  asked  how  their  companies’  radio  communications  practices  affected 
I/E  and  pilot  workload  in  the  simulator  compared  to  instructing/evaluating  pilots  in  the  air  with 
real  radio  communications  (to  own  airplane,  see  Appendix  C,  Question  3).  For  each  type  of 
communication,  ATC  ATIS,  Clearance  Delivery,  and  Company  Dispatch,  Ramp/Gate,  Cabin, 
and  Other,  I/Es  indicated  on  a  scale  from  “1”  to  “5”  whether  the  workload  was  “Lower”  (1)’ 
“Same”  (3),  or  “Higher  than  A/C”  (5). 

ATIS  was  the  only  type  of  information  where  the  majority  of  EEs  indicated  that  it  was  provided 
by  some  kind  of  supporting  system,  depending  on  the  simulator  used.  This  could  consist  of  either 
recorded  or  synthetic  controller  voice  or  printed  handouts.  Since  the  focus  in  this  investigation  is 


7  Only  two  of  these  reported  percentages  for  LOE  (20  percent)  and  SPOT  (15  percent). 
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the  workload  caused  by  I/Es  having  to  role-play  oral  radio  communications,  the  workload 
perceptions  of  I/Es  for  ATIS  are  not  included  in  the  analysis  below. 


Effect  of  Radio  Communications  (To  Own  Airplane)  Role-Play  on  Perceived 
Tnstructor/Evaluator  Workload 

Effect  ofATC  Communications  Role-Play  on  HE  Workload 

Tower  Ground,  Tower  Local,  Approach/Departure  and  ARTCC  ATC  communications  are 
almost  exclusively  simulated  by  the  I/Es  alone,  with  the  exception  of  three  I/Es  from  one  airline 
that  are  supported  by  an  additional  (flight  engineer)  I/E.  These  three  were  excluded  from  the 
analysis  below,  leaving  26  responses  (if  all  others  provided  an  answer). 

■  For  Clearance  Delivery,  only  the  responses  from  21  I/Es  that  still  have  to  role  play  this 
information  alone  were  included  for  the  effect  of  ATC  communications  role-play  on  I/E 
workload.  The  average  I/E  workload  rating  was  3.71,  with  14  I/Es  (67  percent)  judging  it  to 
be  higher,  six  (29  percent)  the  same  and  one  (5  percent)  lower  than  in  the  airplane  (see 
Table  4). 

Table  4.  Effect  and  Importance  of  ATC  Communications 


Communications  with  ATC 

Compared  to 
Actual  Aircraft 
v  Operations: 

Clearance 

Delivery 

Tower/ 

Ground 

Tower/ 

Local 

Approach/ 

Departure 

Center 

Instructor 

Workload1 

3.71 

n  =  21 

3.65 

n  =  26 

3.64 

n  =  25 

3.81 

n  =  26 

3.84 

n  =  25 

;;  ;  : . ■  ' 

Pilot  Workload 

2.86 

n  =  21 

2.46 

n  =  24 

2.60 

n  =  25 

2.56 

n  =  25 

2.50 

n  =  24 

Overall 
Importance  in 
Simulation 

3.39 

n  =  28 

3.86 

n  =  28 

3.77 

n  =  26 

3.92 

n  =  26 

3.69 

n  =  26 

■  For  Tower  Ground  radio  communications,  four  (15  percent)  I/Es  indicated  that  the  I/E 
workload  was  lower  than  in  the  airplane.  Seventeen  (65  percent),  however,  felt  that  the  I/E 
instructor  workload  was  higher  than  in  the  airplane.  Five  perceived  no  difference  between 
airplane  and  simulator  workload.  The  average  response  was  3.65. 

■  For  Tower  Local  communications,  the  average  of  25  responses  (one  I/E  did  not  circle 
anything)  was  3.64.  Four  I/Es  (16  percent)  felt  that  the  workload  in  the  simulator  was  lower 
in  the  simulator,  while  17  (68  percent)  felt  that  it  was  higher.  Four  did  not  perceive  any 
difference. 
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■  For  Approach/Departure,  the  average  of  26  responses  rose  to  3.81.  Three  (12  percent) 
indicated  lower  and  18  (69  percent)  indicated  higher  workload  than  in  the  airplane.  Five 
circled  “3”  for  “same.” 

■  The  perceived  I/E  communications  workload  compared  to  actual  operations  was  rated 
slightly  higher  for  communications  during  the  en  route  phase.  The  average  was  now  3.84, 
with  only  two  (8  percent  of  25  respondents)  indicating  lower  communications  workload  than 
in  the  airplane.  Eighteen  (72  percent)  indicated  higher  and  five,  the  same  workload. 

Effect  of  Company  Communications  Role-Play  on  HE  Workload 

With  the  exception  of  company  dispatch,  where  one  I/E  had  indicated  that  it  was  provided  via 
simulated  ACARS,  all  company  communications  were  reportedly  provided  via  role-play.  These 
included,  in  addition  to  Dispatch  for  most  EEs,  also  Ramp/Gate,  Cabin,  and  Other.  The  three 
EEs  getting  support  from  an  additional  I/E  were  again  excluded. 

■  For  Dispatch,  the  average  of  the  24  ratings  was  3.75.  Eighteen  (75  percent)  respondents 
indicated  that  I/E  communications  workload  was  higher  than  in  the  airplane.  Three  indicated 
that  it  was  the  same,  and  three  that  it  was  lower  (12.5  percent  each,  see  Table  5). 


Table  5.  Effect  and  Importance  of  Company  Communications  and  Partyline 


Communications  with  Company 

Other  Aircraft 

MBM 

Ramp/ 

Gate 

Cabin 

Other 

(Mx) 

Partyline 

3.75 

3.38 

3.61 

3.82 

4.18 

Workload 

n  =  24 

«  =  24 

n  =  23 

n  —  11 

n=  17 

2.83 

2.71 

2.83 

2.88 

n  =  24 

n  =  24 

n  =  23 

71=17 

jjfllM 

3.46 

3.15 

3.82 

3.80 

3.60 

HHMMM 

n  =  28 

n  =  26 

n  =  23 

n=  10 

77  =  24 

■  For  Ramp/Gate,  the  average  of  24  responses  was  3.38.  Thirteen  (54  percent)  perceived  the 
workload  as  higher,  seven  (29  percent)  the  same,  and  four  (17  percent)  as  lower. 

■  For  Cabin  communications,  the  average  I/E  workload  rating  of  23  responses  was  3.61. 
Sixteen  (70  percent)  perceived  the  I/E  workload  as  higher,  three  (13  percent)  as  the  same, 
and  four  (17  percent)  lower. 

■  Eleven  I/Es  circled  a  rating  for  “Other,”  ten  writing  “Maintenance”  in  the  space  provided  and 
one  leaving  it  blank.  The  average  rating  was  3.82,  with  eight  (73  percent)  circling  a  rating 
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that  was  higher,  two  (18  percent)  as  lower  and  one  I/E  (9  percent)  that  the  workload  in  the 
simulator  was  the  same  as  in  the  airplane. 

Effect  of  Radio  Communications  (To  Own  Airplane)  Role-Play  on  Perceived  Pilot  Workload 

While  radio  communications  role-play  may  increase  I/E  workload,  any  lack  of  realistic  radio 

communications  may  decrease  pilot  workload  (Appendix  C,  Question  3). 

Effect  ofATC  Communications  Role-Play  on  Pilot  Workload 

■  For  Clearance  Delivery,  again  only  the  responses  from  the  21 1/Es  that  still  have  to  role-play 
this  information  alone  were  included  for  the  effect  of  ATC  communications  role-play  on 
pilot  workload.  The  average  pilot  workload  rating  was  2.86,  with  two  (10  percent)  of  the  I/Es 
judging  it  to  be  higher  than  in  the  airplane,  the  rest  lower  (four  or  19  percent)  or  the  same  (15 
or  71  percent,  see  Table  4). 

■  In  comparison  to  actual  operations,  Tower  Ground  pilot  communications  workload  was 
perceived  as  lowest.  Ten  of  24  EEs  answering  this  question  perceived  pilot  workload  to  be 
lower  in  the  simulator  than  in  the  airplane  (42  percent).  Thirteen  (54  percent)  perceived  it  to 
be  the  same,  and  only  one  perceived  it  to  be  higher.  The  average  of  the  responses  was  2.46. 

■  Tower  Local  communications  workload  was  rated  by  25  EEs,  resulting  in  an  average  pilot 
workload  of  2.60.  Eleven  (44  percent)  felt  that  the  pilots’  workload  is  lower  in  the  simulator. 
Twelve  (48  percent)  perceived  it  to  be  the  same.  Two  respondents  discerned  it  to  be  higher. 

■  For  Approach/Departure,  the  average  EE  perception  of  pilot  workload  was  2.56.  The 
distribution  of  responses  was  identical  to  Tower  Local,  with  the  same  EEs  perceiving  pilot 
workload  as  lower,  same,  or  higher. 

■  With  an  average  of  2.50,  the  perceived  pilot  communications  workload  compared  to  actual 
operations  was  second  lowest  for  the  en  route  phase.  The  same  1 1  EEs  that  perceived  pilot 
workload  as  lower  for  Tower  Local  and  Approach/Departure  said  so  again  for  ARTCC 
communications.  Twelve  perceived  it  as  the  same,  and  one  as  higher. 

Effect  of  Company  Communications  Role-Play  on  Pilot  Workload 8 

■  For  Dispatch  communications,  the  average  workload  rating  of  24  responses  of  EEs  providing 
dispatch  information  via  role-play  was  2.83.  Whereas  14  (58  percent)  of  EEs  perceived  the 
pilot  workload  for  handling  these  communications  to  be  the  same  in  the  simulator  as  in  the 
airplane,  seven  (29  percent)  discerned  it  to  be  lower.  Three  (13  percent)  perceived  it  as 
higher. 

■  For  Ramp/Gate  communications,  the  average  of  24  responses  was  2.71.  Seventeen  EEs  (71 
percent)  circled  the  number  “3”  indicating  that  pilot  workload  in  the  simulator  was  the  same 


8Given  that  I/Es  consistently  indicated  simulating  company  communications  via  I/E  role  play  (with  one  exception  of 
dispatch),  this  analysis  includes  the  one  instance  each  in  dispatch,  ramp/gate,  and  cabin  where  an  I/E  gave  his 
opinion  without  having  indicated  his  method  of  simulation. 
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as  in  the  airplane.  Six  (25  percent)  felt  that  pilot  workload  was  lower  and  one  (4  percent)  that 
it  was  higher  than  in  the  airplane. 

■  For  Cabin  communications,  the  average  I/E  perception  of  pilot  workload  of  23  responses  was 
2.83.  Twelve  (52  percent)  perceived  the  pilot  workload  the  same  as  in  the  airplane.  Seven  (30 
percent)  perceived  the  pilot  workload  to  be  lower,  while  four  (17  percent)  perceived  it  as 
higher. 

■  For  the  eleven  responses  to  workload  for  “Other”  company  communications  (mainly 
maintenance),  the  average  rating  was  2.09.  Eight  I/Es  (73  percent)  perceived  pilot  workload 
to  be  lower  and  three  (27  percent)  that  it  was  the  same. 

Effect  of  Partvline  Communications  Role-Plav  on  Workload 

I/Es  were  asked  to  compare  the  effect  of  their  “company’s  partyline  simulation  practices”  on  I/E 
and  pilot  workload  to  workload  “in  the  air  with  a  real  airplane”  by  circling  a  number  on  a  scale 
from  “1”  to  “5”.  Again,  (1)  was  labeled  as  “Lower,”  (3)  as  “Same,”  and  (5)  as  “Higher”  (see 
Appendix  C,  Question  6). 

Four  I/Es  reported  that  they  did  not  simulate  the  “partyline,”  and  four  additional  I/Es  did  not 
answer  this  question.  After  exclusion  of  the  three  I/Es  receiving  support  from  other  instructors 
and  one  VE  who  reported  provision  of  a  recorded  partyline  (presumably  GATES,  see 
Technology  Review),  17  respondents  remain,  whose  responses  are  summarized  below. 

■  For  perceived  I/E  workload,  the  average  is  4.18,  with  76  percent  or  13  of  the  respondents 
perceiving  it  as  higher  in  the  simulator  than  in  the  air.  Three  (18  percent)  perceive  it  as  the 
same  and  one  (6  percent)  as  lower  (see  Table  5). 

■  Pilot  workload  was  perceived  to  be  slightly  lower  than  in  the  air,  with  an  average  rating  of 
2.88.  Six  I/Es  (35  percent)  each  perceive  it  to  be  lower  or  the  same.  Five  (29  percent) 
perceive  it  to  be  higher. 

3.2.2  Overall  Importance  of  Simulating  Radio  Communications  for  Training 
Effectiveness 

Two  more  questions  tapped  I/Es  perception  of  the  importance  of  realistically  simulating  radio 
communications.  One  asked  about  the  importance  of  radio  communications  in  scenarios  aiming 
at  training  specific  types  of  skills  such  as  ATC  procedures,  CRM,  distraction  management,  and 
situation  awareness  (Appendix  C,  Question  7).  The  other  asked  about  importance  of  each  type  of 
ATC  and  company  communications  in  the  context  of  overall  training  effectiveness  (Appendix  C, 
Question  3). 

Importance  of  Radio  Communications  in  Scenarios 

I/Es  were  asked  to  indicate  how  frequently  they  “rely  on  radio  communications  as  part  of  the 
scenario  context  for  team  performance”  for  each  type  of  skill  indicated  below.  The  scale  ranged 
from  “Never”  (1)  to  “Sometimes”  (3)  to  “Often”  (5).  They  were  also  asked,  for  each  skill,  to 
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indicate  the  importance  of  radio  communications  for  effectively  training  this  skill,  on  a  scale 
ranging  from  (1)  “Very  Low”  to  (5)  “Very  High.” 

All  I/Es  offered  their  opinion  on  some  of  these  questions.  As  shown  in  Table  6,  the  frequency 
with  which  skills  listed  were  viewed  as  relying  on  radio  communications  fell  between 
“sometimes”  and  “often”  (ranging  from  3.58  to  3.89).  In  addition,  radio  communications  were 
consistently  perceived  as  important  for  effective  training  scenarios  (ranging  from  3.77  to  4.17) 
with  little  variation  across  skills. 

Table  6.  Frequency  and  Importance  of  Radio  Communications  for  Skills  Training 


: 

■  .  1  ■-  ■  -  ■  ;  .  ■:  .  ; 

Type  of  Skills 
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■  For  “New  ATC  Procedures,  e.g.,  PRM”  (Parallel  Runway  Monitored  Approach),  the  average 
frequency  rating  of  the  26  I/Es  responding  was  3.58.  Twenty-four  (92  percent)  of  I/Es  circled 
“3”  indicating  sometimes  or  higher.  Only  two  indicated  that  this  type  of  skill  rarely  relies  on 
radio  communications  as  part  of  the  scenario  by  circling  “2”.  The  average  rating  for  the 
importance  of  radio  communications  for  training  new  ATC  procedures  effectively  was  3.77, 
with  16  I/Es  (62  percent)  indicating  the  importance  as  high  (4)  or  very  high  (5).  Eight  (31 
percent)  indicated  “3”  and  two  (8  percent)  circled  “2”. 

■  For  “Non-Routine  ATC  (e.g.,  pilot/ ATC  coordination),”  the  average  frequency  rating  of  28 
respondents  was  3.89.  Twenty-one  (75  percent)  circled  “4”  or  “5”,  five  (18  percent)  circled 
“3”  and  two  circled  “1”  or  “2”.  The  average  rating  for  the  importance  of  radio 
communications  for  training  non-routine  ATC  was  3.96,  with  24  (86  percent)  indicating  high 
or  very  high.  One  I/E  (4  percent)  circled  “3”  and  three  (1 1  percent)  thought  the  importance 
was  low  or  very  low. 
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■  For  “CRM:  Crew  Coordination  (e.g.,  cabin,  dispatch,  maintenance),”  the  average  frequency 
rating  of  29  respondents  was  3.72.  Nineteen  (66  percent)  circled  “4”  or  “5”  and  nine  (31 
percent)  sometimes.  One  circled  “2”.  One  I/E  commented  that  “[i]f  CA  [captain]  does  not 
delegate  duties,  my  technique  is  to  load  the  crew  up  with  B.S.  [sic]  radio  transmissions.”  For 
importance  of  radio  communications  for  CRM  training,  the  average  of  28  responses  was 
4.17,  the  highest  average  rating  for  the  type  of  skills  queried.  Twenty-five  (89  percent) 
indicated  that  the  importance  was  high  or  very  high,  two  (7  percent)  were  neutral  and  one 
circled  “2.” 

■  The  average  frequency  rating  for  “Distraction  Management”  of  28  responses  was  3.64. 
Sixteen  (57  percent)  responded  by  circling  “4”  or  “5”,  seven  (25  percent)  indicated 
“sometimes,”  and  five  “rarely”  or  “never.”  With  regard  to  the  importance  of  radio 
communications  simulation  for  distraction  management  training,  the  average  of  27  responses 
was  3.93.  Twenty-one  (78  percent)  indicated  that  it  was  high  or  very  high,  three  were  neutral 
and  three  indicated  that  it  was  low  or  very  low  (1 1  percent  each). 

■  For  “Situation  Awareness/Partyline”  skills,  the  average  frequency  rating  of  27  responses  was 
3.67.  Fifteen  (56  percent)  circled  “4”  or  “5”,  nine  (33  percent)  indicated  sometimes,  and 
three  (11  percent)  rarely.  With  regard  to  importance  of  radio  communications  for  effective 
training  of  situation  awareness  via  “partyline ,”  the  average  of  28  responses  was  3.89,  with  19 
FEs  (68  percent)  perceiving  it  as  high  or  very  high,  seven  (25  percent)  as  medium,  and  two 
as  low. 

Overall  Importance  of  Radio  Communications 

The  question  of  overall  importance  for  training  effectiveness  of  “realistically  simulating  each 
type  of  radio  communications”  was  asked  in  conjunction  with  the  question  on  the  effect  of  radio 
communications  simulation  on  I/E  and  pilot  workload  (Appendix  C,  Question  3).  The  scale 
provided  for  circling  was  “1”  to  “5”  with  (1)  labeled  as  “Very  Low”  and  (5)  as  “Very  High.” 

When  considering  the  results  to  this  question,  one  caveat  should  be  kept  in  mind.  When  I/Es 
were  asked  for  their  opinions  on  workload,  they  were  explicitly  asked  to  consider  “your 
company’s  radio  communications  simulation  practices.”  In  this  subquestion,  which  was  part  of 
the  same  question,  they  were  asked  about  the  importance  of  “realistically  simulating”  the 
different  types  of  radio  communications  on  overall  importance  of  radio  communications 
regardless  of  their  opinion  on  the  effectiveness  on  their  company’s  practices.  Because  this 
distinction  was  not  explicitly  stated,  the  answers  of  some  I/Es  considering  their  company’s 
practices  as  ineffective  may  have  inflated  the  number  of  low  ratings  for  overall  importance.  For 
example,  one  I/E  who  had  consistently  rated  the  importance  of  ATC  communications  as  low  and 
company  communications  as  very  low,  added  a  comment  that  these  communications  are  “not 
very  effective  during  simulation,  since  the  instructor  must  cover  all  bases  himself.”  The  ratings 
of  this  I/E  were  not  included  in  the  ATC  and  company  communications  importance  ratings 
below,  because  his  opinions  were  clearly  better  reflected  in  his  answers  on  radio 
communications  in  scenarios  (see  above).  There,  he  gave  high  ratings  of  four  to  the  importance 
of  radio  communications  for  training  new  ATC  procedures  and  non-routine  ATC,  adding  the 
comment  that  “[r]adiocommunications  is  the  most  effective  way  to  simulate  different  kinds  of 
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approaches  (e.g.,  ILS  approach  but  glideslope  out,  localizer  only  approach).”  All  other  responses 

from  this  I/E  were  included.9 

ATC  Communications  to  Own  Airplane 

■  The  average  of  28  ratings  of  the  importance  of  realistically  simulating  ATIS  for  training  and 
evaluation  was  3.54.  Fourteen  (half)  of  I/Es  responded  high  or  very  high,  1 1  (39  percent) 
rated  it  “3”,  and  three  (1 1  percent)  rated  it  “2”  (see  Table  4). 

■  For  Clearance  Delivery,  an  average  of  28  ratings  for  importance  was  3.39.  Twelve  (43 
percent)  each  circled  “4”  or  “5”  and  12  circled  “3”,  while  four  (14  percent)  circled  “2”. 

■  The  importance  of  simulating  Tower  Ground  communications  was  perceived  as  high  or  very 
high  by  19  or  68  percent  of  28  raters.  Seven  (25  percent)  indicated  medium  importance  by 
circling  “3”,  and  two  (7  percent)  circled  “2”.  The  average  rating  was  3.86. 

■  On  average,  the  importance  of  Tower  Local  was  perceived  as  3.77  by  26  raters.  Seventeen 
(65  percent)  considered  it  as  high  or  very  high,  seven  (27  percent)  as  medium  (3),  and  two  (8 
percent)  considered  it  as  low  (2  percent). 

■  The  average  importance  rating  of  26  respondents  for  realistic  Approach/Departure 
communications  simulation  was  3.92.  Nineteen  (73  percent)  felt  its  importance  was  high  or 
very  high.  Six  (23  percent)  gave  it  the  medium  rating  of  “3”,  and  one  (4  percent)  circled  “2”. 

■  For  importance  of  Center  communications,  of  26  responses,  the  average  was  3.69.  Sixteen 
(61  percent)  responded  that  Center  communications  importance  was  high  or  very  high.  Eight 
(31  percent)  gave  it  a  neutral  rating.  Two  (8  percent)  indicated  that  its  importance  was  low. 

Company  Communications  to  Own  Airplane 

■  For  importance  of  realistically  simulating  Dispatch  communications  for  training  and 
evaluation,  the  average  of  28  responses  was  3.46.  Twelve  (43  percent)  each  perceived  its 
importance  as  high  or  very  high,  (circling  “4”  or  “5”)  or  medium,  (circling  “3”).  Four  (14 
percent)  circled  “2”  (see  Table  5). 

■  Ramp/Gate  communications  were  given  an  average  importance  rating  of  3.15  by  26 
respondents.  Seven  (27  percent)  circled  high  or  very  high.  Fifteen  (58  percent)  circled  “3”  for 
medium.  Four  (15  percent)  circled  low  or  very  low. 

■  For  communications  with  cabin  personnel,  the  average  importance  rating  of  23  responses 
was  3.83.  Thirteen  (57  percent)  responded  high  or  very  high.  Ten  (43  percent)  responded  that 
its  importance  was  medium. 


9  In  conjunction  with  these  same  inquiries,  I/Es  were  also  asked  about  training  effectiveness  of  radio 
communications  in  simulations.  Unfortunately,  the  wording  of  this  question  failed  to  distinguish  whether  I/Es 
should  answer  this  question  for  their  method  of  providing  radio  communications,  or  for  realistic  radio 
communications  in  general,  as  they  were  asked  for  overall  importance.  Due  to  this  ambiguity,  the  results  for  training 
effectiveness  in  Question  three  are  not  included  here. 
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■  Ten  I/Es  rated  the  importance  of  “Other”  company  communications,  most  filling  in 
“maintenance”  and  two  leaving  it  blank.  Six  circled  “4”  or  “5”  for  high  or  very  high,  and 
four  circled  “3”.  The  average  rating  was  3.80. 

Partyline  Communications 

■  Twenty-five  I/Es  rated  the  importance  of  the  partyline  for  training  and  evaluation,  giving  it 
an  average  rating  of  3.60.  Sixteen  (64  percent)  believe  its  importance  to  be  high  or  very  high. 
Five  (20  percent)  gave  it  a  medium  rating  (3  percent).  Four  (16  percent)  believe  its 
importance  to  be  low  or  very  low. 

3.2.3  Anticipated  Effect  of  Automating  Radio  Communications 

A  last  question  asked  I/Es  “what  types  of  radio  communications  you  would  like  to  see  automated 

and  what  do  you  think  the  effect  would  be  on  instructor  workload  and  training  effectiveness.” 

For  each  of  the  four  types  of  radio  communications  offered,  the  five-point  scale  ranged  from  (1) 

labeled  “Lower”  to  (3)  “Same”  to  (5)  “Higher”  than  what  it  is  now  (see  Appendix  C,  Question 

8b). 

■  Twenty-five  instructors  responded  to  “Frequency  change/handoffs.”  The  average  workload 
rating  was  3.04.  Five  (20  percent)  responded  that  the  I/E  workload  would  be  the  same,  and 
an  equal  number  of  10  (40  percent  each)  gave  ratings  above  and  below  “3”.  For  “Training 
Effectiveness,”  the  average  of  24  responses  was  3.58,  with  13  (54  percent)  rating  it  as  higher 
than  now,  eight  (33  percent)  as  the  same,  and  only  three  (12.5  percent)  as  lower  (Table  7). 

Table  7.  Effect  of  Automated  Radio  Communications  on  Workload  and  Training 
Effectiveness 
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■  With  regard  to  “Clearances  to  own  airplane,”  the  average  rating  of  25  for  1/E  workload  was 
3.00.  Eleven  (44  percent)  of  I/Es  anticipated  I/E  workload  to  be  lower,  four  (16  percent)  the 
same,  and  10  (40  percent)  higher.  Again,  I/Es  felt  that  training  effectiveness  would  improve, 
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giving  it  an  average  rating  of  4.00.  Nineteen  (76  percent)  anticipated  that  it  would  be  higher, 
five  (20  percent)  same,  and  only  one  (4  percent)  that  it  would  lower  than  now. 

■  If  “Clearances  to  other  aircraft”  were  automated,  26  I/Es  anticipated  an  average  workload  of 
2.88.  Ten  (38  percent)  gave  ratings  lower  than  “same,”  four  (15  percent)  as  same,  and  12  as 
higher  (46  percent).  With  an  average  rating  of  3.85,  most  I/Es  felt  that  training  effectiveness 
would  be  improved.  Nineteen  (73  percent)  anticipated  it  to  be  higher,  four  (15  percent)  the 
same,  and  three  (12  percent)  lower. 

■  For  ACARS,  the  average  anticipation  of  I/E  workload  of  21  responses  was  3.05.  Seven  or  a 
third  anticipated  it  to  be  lower,  five  (24  percent)  the  same,  and  nine  (43  percent)  higher. 
Training  effectiveness  got  the  highest  average  rating  of  all  options  offered,  4.30  for  20 
responses.  Sixteen  (80  percent)  anticipated  it  to  be  higher  and  four  (20  percent)  the  same. 

Several  wrote  in  suggestions  in  the  Comment  space  provided.  These  included  “ATIS”  (eight 
votes),  and  “Cabin,”  “ATC  chatter,”  and  “Company  via  ACARS”  with  two  votes  each. 
“Dispatch”  was  written  in  once. 

3.3  SURVEY  SUMMARY  AND  ADDITIONAL  INSTRUCTOR/EVALUATOR 
COMMENTS 

Twenty-nine  I/Es  participated  from  14  different  airlines.  All  but  one  of  these  indicated 
participating  in  LOFT,  20  in  LOEs,  and  19  in  SPOT.  Both  ATC  and  company  communications 
reportedly  were  most  faithfully  simulated  during  LOFT,  followed  by  LOE.  Not  surprisingly, 
SPOT  ranked  last,  since  it  relies  least  on  full  mission  simulation. 

All  I/Es  queried  indicated  simulating  some  ATC  communications,  especially  in  the  terminal 
environment.  For  company  communications,  all  but  one  of  the  I/Es  reported  simulating  at  least 
some  of  them,  most  frequently  dispatch. 

Regarding  simulation  of  other  traffic,  almost  30  percent  of  participants  did  not  report  simulating 
other  traffic,  whether  out-the-window,  on  TCAS,  or  via  partyline  communications.  Almost  60 
percent  reported  simulating  some  traffic  out-the-window  on  the  airport  surface,  but  less  than 
one-third  in  the  terminal  and  en  route  environments.  A  good  third  reported  simulating  TCAS. 

With  regard  to  the  partyline,  less  than  40  percent  reported  providing  communications  to  or  from 
other  traffic,  mainly  on  the  surface.  Only  two  of  these  provide  terminal  and  en-route  partyline 
information.  “[N]one  of  our  formal  training  documentation  requires  it,”  wrote  one  of  the  I/Es, 
and  “very  seldom  used  due  to  I/E  workload,”  another.  One  I/E  also  wrote  that  it  is  “rarely  used” 
unless  it  serves  a  specific  purpose:  “Sometimes  the  I/E  will  clear  other  aircraft  into  position  to 
effect  a  rejected  landing  or  a  TCAS  event.”  This  was  confirmed  by  another,  who  wrote  that  he 
simulates  the  partyline  “very  little,  we  may  clear  the  aircraft  at  the  hold  line  for  take-off  as  a  cue 
that  we  are  next  to  take  off.” 
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Most  radio  communications  are  simulated  by  I/E  role-play,  with  few  exceptions.  I/Es  from  one 
carrier  instructing  a  three-person  cockpit  report  that  two  I/Es,  one  for  the  Captain/First  Officer 
and  one  for  the  flight  engineer,  share  communication  duties.  Five  I/Es  indicated  using  handouts 
or  ACARS  for  Clearance  Delivery,  and  over  half  reportedly  use  either  printed  or  recorded 
(synthetic  or  natural  voice)  ATIS. 

Regarding  their  workload  in  the  simulator,  I/Es  reported  that  for  LOEs  and  LOFT,  they  spend  at 
least  half  of  their  time  and  effort  observing.  In  LOFT,  they  spend  8  percent  of  their  time/effort 
instructing  versus  only  2  percent  in  LOEs.  This  makes  sense  because  the  latter  is  an  evaluation 
event.  In  SPOT,  they  spend  a  quarter  of  their  time/effort  instructing,  at  the  cost  of  observing 
(only  36  percent).  Radio  communications  simulation  takes  between  20  to  22  percent  for 
LOFT/LOE  and  16  percent  for  SPOT.  Managing  the  simulator  systems  requires  a  consistent  21 
to  22  percent  of  I/E  time  and  effort. 

One  I/E  volunteered  the  information  that  he  spends  20  percent  of  his  time  and  effort  filling  out 
forms,  while  another  indicated  it  was  5  percent  of  his  time/effort.  Although  this  activity  was  not 
included  in  the  questionnaire,  it  is  probably  safe  to  assume  that  all  I/Es  spend  some  effort  filling 
out  forms.  This  may  further  reduce  their  ability  to  provide  an  adequate  level  of  radio 
communications. 


Instructor/ evaluators  were  polled  on  their  opinion  on  the  effect  of  having  to  simulate  radio 
communications  on  workload  and  training  effectiveness,  as  well  as  on  the  importance  of 
simulating  the  partyline.  They  were  also  asked  for  their  opinions  on  the  overall  effect  of 
realistically  simulating  radio  communications  in  scenarios. 

Most  felt  that  I/E  impersonation  of  communications  to  own  airplane  increases  the  instructor 
workload  and  “divides  his  attention.”  One  I/E  added  that  this  is  especially  difficult  for  relatively 
new  instructors.  On  the  other  hand,  it  reduces  pilot  workload  by  decreasing  the  communications 
load  compared  to  the  real  world,  especially  company  communications.  They  feel  that  “company 
communications  are  not  normally  used  [in  simulation],  too  time-consuming.”  Another  wrote  that 
I/E  can  become  task  saturated  when  crew  works  two  VHF  radio  and/or  communicates  with 
cabin  simultaneously.”  One  mentioned  that  I/E  impersonation  of  radio  communications  even 
reduces  the  manual  workload  of  pilots,  because  they  normally  do  not  need  to  redial  a  new 
frequency  to  communicate  in  the  simulator. 

I/Es  generally  feel  that  the  partyline  is  greatly  reduced  when  the  task  of  simulating  it  remains 
with  the  PE,  thus  reducing  pilot  workload  further.  They  think  that  the  presence  of  meaningful 
partyline  information  would  increase  training  effectiveness,  by  “teaching  pilots  to  listen”  and 
“enabling  [I/Es]  to  assess  CRM  elements  such  as  workload  and  distraction.”  “There  needs  to  be 
extemporaneous  background  communication  between  ATC  and  other  aircraft,”  one  I/E  stated, 
and  another  added  that  the  partyline  is  his  “biggest  concern,  so  pilots  are  listening.”  One  more 
PE  referred  to  the  importance  of  the  partyline  for  helping  to  simulate  a  “realistic  crowded 
frequency.” 


PEs  indicated  that  scenarios  rely  on  radio  communications  simulation  to  teach  such  skills  as 
(new)  ATC  procedures,  CRM,  and  situation  awareness  effectively.  The  overall  importance  of 
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radio  communications,  for  ATC,  is  perceived  highest  for  the  terminal  environment.  For 
company,  it  may  be  cabin  and  maintenance,  although  the  latter  may  be  simulated  only  “if  a 
particular  procedure  might  require  maintenance  help.” 

I/E  ratings  indicated  skepticism  with  respect  to  the  success  of  automating  radio  communications 
and  its  effect  on  reducing  workload  (one  I/E  commenting  “No  way — the  simulator  periods  are 
too  dynamic!”).  Yet,  they  feel  that  it  would  increase  training  effectiveness,  especially  for 
clearances  to  other  aircraft.  Perhaps  due  to  I/Es  skepticism  regarding  other  kinds  of 
communication  automation,  ACARS  and  ATIS  received  the  highest  rankings  for  the  anticipated 
effect  of  automation  on  training  effectiveness. 

In  addition  to  the  one  I/E  referring  to  the  effect  of  instructor  experience  on  radio 
communications  role-play,  one  more  I/E  alluded  to  the  differences  in  scenario  complexity  and 
workload  induced  by  instructor,  which  is  “all  relative  to  the  creative  ability  of  the  instructor.” 

Several  I/Es  made  general  comments  on  how  radio  communications  are  a  “critical  need  for  LOS 
realism.”  “There  is  a  correlation  between  realism  in  communications  and  training  effectiveness; 
better  training  results  from  more  realism.”  One  I/E  concluded  his  survey  with  the  remarks:  “This 
is  a  subject  near  and  dear  to  my  heart.  [...]  I  believe  the  ‘simulator  mind-set  syndrome’  must  be 
fought  with  realism.  How  can  we  expect  crews  to  ‘treat  simulator]  like  the  aircraft’  when  the 
audio  environment  belies  the  condition  so  often?” 
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4.  TECHNOLOGY  REVIEW 


Regulation,  market  forces,  and  airlines’  own  desire  for  improvements  are  driving  forces  for 
simulator  technology  advancement.  The  need  for  realistic  radio  communications  including 
simulation  of  the  partyline  is  mentioned  both  in  the  AQP  and  CRM  Advisory  Circulars  (FAA, 
AC120-54A  and  AC120-35B,  see  Literature  Review  section).  The  Instructor/Evaluator  Task 
Listing  Training  Level  Performance  Statements  (FAA,  2000)  says  that  I/Es  “advise  crew 
members  that  LOS  presents  ‘real  world’  problems  and  the  Instructor  will  role  play  ATC,  F/A, 
Dispatcher,  etc.”  and  that  “crew  members  should  prepare  and  act  as  if  on  the  line.” 

One  example  of  airline  development  of  improved  radio  communications  simulation  is  the  United 
Airlines  IRAS  or  “Interactive  Real  Time  Audio  System,”  also  (and  somewhat  too  dismissively) 
known  as  “Chatter  Program.”  United  invested  in  a  multi-year  in-house  program  to  attempt  to 
modify  many  of  its  current  simulators  to  provide  a  more  realistic  radio  communications 
environment. 

CAE  Electronics  Ltd.  described  market  forces  as  driving  the  development  of  their  GATES  or 
“Ground  and  Air  Traffic  Environment  System”  product.  A  customer  specification  of  a  more 
realistic  airport  and  terminal  environment  was  the  driving  force  for  GATES  development. 
Another  example  of  the  market  clearly  perceiving  the  value  of  realistic  radio  communications  is 
the  recent  proliferation  of  Personal  Computer  Commercial  Off-The-Shelf  (PC  COTS)  systems. 
Personal  Computer  Aviation  Training  Devices  (PCATD)  and  entertainment  flight  simulator 
products  provide  interesting  approaches  to  simulating  the  traffic  environment. 

Intelligent  systems,  which  have  been  mainly  developed  for  military  applications,  have  the 
potential  for  a  high  degree  of  communication  realism.  TacAir  SOAR,  e.g.,  simulates  pilot 
behavior  and  decision  making  in  complex  airborne  tactical  warfare  distributed  simulations. 
Intelligent  systems  may  be  the  most  promising  for  the  future,  although  a  specific  ATC 
implementation  remains  to  be  developed  (see  Section  4.4). 

The  information  in  this  section  was  gathered  at  site  visits,  through  interviews  of  users  and 
manufacturers,  and,  for  some  of  the  PC  COTS  systems,  by  purchasing  and  testing  the  system. 

4.1  INTERACTIVE  REAL  TIME  AUDIO  SYSTEM 

IRAS  is  a  United  Airlines  in-house  development  with  very  high  operational  realism.  It  is  based 
on  field  recordings  of  actual  ATC  communications  on  UAL  routes.  By  dubbing  ATC  with  the 
respective  I/E  voice,  the  I/E  can  intervene  without  the  pilot  trainee  realizing  it.  The  engineers  at 
United  attempted  to  include  many  subtle  nuances  of  audio  communications  into  the  environment 
including  ATC  driven,  demanding  timing,  frequency  congestion,  foreign  or  regional  accents, 
stuck  microphones  blocking  an  entire  frequency,  and  meaningful  partyline  to  train  pilots  to  listen. 

The  system,  however,  encountered  many  technical  difficulties.  Algorithms  coded  to  trigger  ATC 
or  partyline  recordings  at  appropriate  times  did  not  always  function  adequately,  degrading 
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realism  and  forcing  an  embarrassed  instructor  to  intervene,  increasing  instructor  stress  and 
workload.  Operations  in  dense  terminal  areas  also  taxed  the  scenario  algorithms,  with  normal 
variations  in  crew  response  causing  additional  timing  problems  and  workload  for  the  instructor. 

The  IRAS  interface  was  often  not  well  integrated  into  the  instructor  station  resulting  in  poor 
ergonomics  and  human  factors.  Different  instructor  interfaces  across  simulator  models  often 
made  it  even  harder  for  instructors  to  operate  the  system.  The  cost  and  difficulty  of  systemwide 
implementation  was  further  hampered  by  different  simulator  designs  with  different  visual,  audio, 
and  navigation  models. 

All  of  the  above  in  combination  with  expensive  scenario  development  including  field  recording, 
transcribing,  dubbing,  database  maintenance,  as  well  as  costly  route,  sector  map,  and  simulator- 
interface  code  development  contributed  to  the  program  gradually  losing  support.  Currently,  the 
program  has  been  scaled  back  to  use  only  one  generic  ATC  voice,  and  it  is  only  used  for  new- 
hire  screening  in  a  727  flight  training  device.10  The  applicants  are  presented  with  relevant 
partyline  information  to  evaluate  their  planning  abilities  during  an  approach,  as  well  as 
automated  frequency  changes.  New  hire  evaluators  reportedly  complain  when  the  system  is  down 
because  of  the  increased  workload  of  providing  frequency  changes. 

The  lessons  learned  from  IRAS  are  that  for  a  system  to  be  successful,  it  must  be  flexible, 
transparent,  easy  to  use,  easy  to  implement  and  maintain,  and  easy  to  integrate  with  different 
scenarios,  simulators,  and  simulator  systems  (e.g.,  visual,  audio,  etc.).  There  may  be  a 
cost/benefit  trade-off  for  the  different  aspects  of  realism  required  for  different  training  and 
evaluation  events  that  must  be  examined  in  this  research.  For  instance,  it  may  not  be  necessary  to 
conceal  instructor  intervention  from  the  pilots,  a  capability  achieved  by  IRAS,  but  at  a  high  price. 

4.2  GROUND  AND  AIR  TRAFFIC  ENVIRONMENT  SYSTEM 

Airlines  have  advanced  the  state-of-the-art  by  specifying  requirements  to  the  simulator 
manufacturers.  The  CAE  product  GATES  came  to  be  following  a  request  from  a  foreign  airline 
to  provide  a  visual  representation  of  traffic  in  the  airport  terminal  environment.  It  soon  became 
obvious  to  the  developers  and  airline  that  correlated  and  meaningful  radio  communications 
would  have  to  be  an  essential  component  of  such  traffic  representation. 

Several  domestic  airlines  and  training  facilities  as  well  as  foreign  airlines  and  military  are 
currently  installing  GATES-equipped  simulators.  The  product  as  it  is  today  provides  simulated 
aircraft  traffic  and  associated  relevant  communications  to  and  from  those  aircraft  on  the  airport 
and  in  the  terminal  environment.  Runway  incursions  can  be  seen  as  well  as  heard.  The  product 
also  provides  voice  simulations  of  ground  traffic  from  emergency  equipment  as  well  as 
maintenance,  fuel,  baggage,  and  aircraft  handling. 

GATES-simulated  vehicles  do  not  follow  scripted  scenarios.  A  continuous  flow  of  arriving  and 
departing  traffic  is  generated.  The  traffic  elements  are  aware  of  and  will  react  to  each  other  and 
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United  Airlines  discussed  plans  to  install  this  audio  system  on  a  DC- 10  simulation  for  new-hire  screening. 
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the  simulated  own  airplane.  The  I/E  has  limited  ability  to  control  specific  situations  like  runway 
incursions  and  emergency  vehicles.  The  I/E  still  provides  all  ATC  communication  to  own 
airplane,  however.  Pre-departure  clearances  are  provided  by  simulated  ACARS. 

The  main  reason  a  domestic  airline  cited  for  choosing  GATES  was  to  add  realism  and  increase 
distraction  factors  for  LOFT  sessions.  Specifically,  they  wanted  to  enhance  TCAS  scenarios, 
ground  hazard  avoidance,  and  runway  incursions.  A  senior  check  airman  responsible  for  GATES 
implementation  stated  that  the  improved  overall  sense  of  realism  alone  justified  its  purchase  and 
support  costs.  Workload  increase  for  the  I/E  is  reportedly  minimal.  He  added  that 
interactive/reactive  communications  capability  would  enhance  usability  and  realism. 

4.3  PERSONAL  COMPUTER  COMMERCIAL  OFF-THE-SHELF  SYSTEMS 

The  PC  COTS  flight  simulator  manufacturers  clearly  perceive  a  market  value  for  including 
realistic  radio  communications  add-ons  to  their  products.  Evaluations  included  a  PCATD 
product  approved  for  limited  credit  in  the  presence  of  a  flight  instructor  (FAA,  1997)  and  three 
PC-based  entertainment  software  systems.  Though  not  approved  for  training  credit,  the  latter  are 
marketed  as  ATC  and  instrument  flight  environment  enhancements  to  instrument  pilots.  With 
respect  to  realistic  radio  communications,  the  methods  of  generating  and  presenting  ATC  and/or 
partyline  communications  as  well  as  the  perceived  limitations  and  strengths  of  the  design  are 
described.  "  ° 


Typical  features  include  concatenated  recorded  natural  speech  to  own  airplane  and  to/from  other 
aircraft,  partyline  consisting  of  random  chatter  with  no  relevance  for  situation  awareness  and  no 
visual  traffic,  and  scenarios  scripted  with  a  simple  decision  tree. 

Flexibility  and  controllability  vary  greatly  among  products.  The  different  systems  add  varying 
degrees  of  realism  and  fun  to  the  simulations,  dependent  on  ease  of  use.  They  are  summarized 
below,  starting  with  the  most  significant  one. 


4.3.1  Jeppesen  FS-200/Flightpro 

FS-200/Flightpro  is  available,  when  packaged  with  the  proper  hardware,  as  an  FAA  approved 
PCATD,  offering  some  credit  in  the  presence  of  a  certified  instrument  flight  instructor  (CFII) 
under  AC61-126  (FAA,  1997).  Specifically,  it  can  be  used  to  meet  the  requirement  of  an 
instrument  rating  in  lieu  of,  but  not  more  than,  10  hours  of  time  that  ordinarily  may  be  acquired 
in  an  aircraft,  flight  simulator,  or  flight  training  device.  According  to  a  Jeppesen  representative, 
the  PCATD  version  is  marketed  toward  flight  schools  as  a  low  cost  alternative  simulator.  The 
FS-200  and  ATC  add-on  software  are  generally  marketed  toward  private  pilots  as  a  tool  to 
provide  a  more  realistic  simulation  to  maintain  currency.  About  50  percent  of  FS-200  owners 
purchase  ATC  packages.  However,  they  are  not  flexible  enough  for  flight  schools,  which  rely  on 
the  instructor  for  simulating  ATC.  The  ATC  flight  packages  are  low  cost-scripted  scenarios. 
They  add  natural  sounding  concatenated  ATC  audio  recordings.  The  partyline  is  relevant.  For 
example,  an  aircraft  ahead  is  being  given  hold  instructions  due  to  weather  and  soon  the  pilot 
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receives  something  similar.  The  script  is  identical  for  every  flight  and  ATC  communications  are 
aborted  if  the  pilot  strays  off  course. 


4.3.2  FlightSafety  International  Aviator  PRO  98 

Aviator  PRO  98  adds  ATC  scripts  to  Microsoft’s  MS  Flight  Simulator  that  function  much  like 
the  Jeppesen  add-on  scripts.  The  inflexible  scripts  do  not  permit  deviations.  One  cannot  request 
any  changes  from  ATC.  The  script  will  terminate  after  a  set  number  of  deviations.  ATC 
clearances  are  recorded  natural  speech.  The  partyline  chatter  is  random  and  not  relevant  to  the 
current  situation,  but  are  rich  in  features  such  as  disfluencies,  distortions,  and  pilots  interrupting 
an  ongoing  exchange  by  keying  their  microphone. 


4.3.3  Proflight 

Proflight  also  adds  features  to  MS  Flight  Simulator.  Its  interface  generates  a  flight  plan,  weather, 
and  ATC  script.  The  scripts  are  somewhat  more  flexible  than  those  of  the  previous  PC  COTS 
systems.  Proflight  accepts  50  different  keyboard  commands  to  acknowledge  ATC  and  make 
requests.  For  example,  a  pilot  can  request  a  different  altitude,  declare  a  missed  approach,  or 
request  vectors.  ATC  clearances  are  concatenated  recorded  natural  speech.  Partyline 
communications  are  limited  to  aircraft  checking  in  with  headings  and  altitudes  and  are  not 
relevant  to  the  current  flight. 

4.3.4  Flight  Unlimited 

Flight  Unlimited  is  entertainment  software  that  offers  a  non-scripted  intelligent  air  traffic  control 
system  with  vectoring  abilities.  There  is  a  relevant  partyline  that  corresponds  to  visual  traffic  in 
common  situations.  It  has  limited  ability  in  some  situations  to  interact  with  other  traffic  visually 
with  correct  communications  to  and  from  other  aircraft.  A  pop-up  menu  provides  the  input 
choices,  possible  responses  and  requests  to  ATC  partyline  and  ATC  clearances  are  concatenated 
recorded  natural  speech. 

In  conclusion,  all  but  one  of  these  systems  are  scripted,  which  represents  similar  limitations  here 
as  encountered  with  IRAS.  COTS  products  that  are  based  on  scripts  limit  users  to  whatever 
scenarios  they  purchased.  They  are  often  not  tolerant  of  deviations  from  the  script,  even  if  these 
deviations  involve  accepted  and  sometimes  even  recommended  navigation  practices.  Pilots  must 
make  requests  from  and  respond  to  ATC  via  keyboard  input,  increasing  their  workload.  The 
introduction  of  even  basic  artificial  intelligence  capabilities  such  as  in  Flight  Unlimited  greatly 
improves  realism  and  flexibility  of  the  system,  significantly  reducing  user  frustration. 


4.4  INTELLIGENT  SYSTEMS 

Intelligent  systems,  which  were  mainly  developed  for  military  applications,  have  the  potential 
for  a  high  degree  of  communication  realism  with  no  instructor  intervention.  They  use  rule-based 
models  of  goals  and  behavior  of  the  speaker  (e.g.,  controller).  Messages  are  generated  based  on 


30 


speaker  task  goals  and  situation  awareness.  This  eliminates  the  need  for  complex  “triggering” 
algorithms. 

This  kind  of  modeling  has  been  successfully  applied  in  manned  and  unmanned  simulations. 
Rapid  Intelligent  Tutoring  Development  Shell  (RIDES,  Flemming  and  Horwitz,  1996)  provides 
an  intelligent  advisor  for  simulator  and  airborne  pilot  training.  RIDES  is  a  tool  for  developing 
agent  behavior  and  permits  the  author/instructor  with  little  or  no  programming  experience  to 
quickly  modify  parts  of  the  simulation. 

TacAir  SOAR  (Jones,  Laird,  Nielsen,  Coulter,  Kenny,  and  Koss,  1999)  simulates  pilot  behavior 
and  decision-making  in  complex  airborne  tactical  warfare  distributed  simulations.  It  integrates  a 
wide  variety  of  intelligent  capabilities  including  real-time  hierarchical  execution  of  complex 
goals,  communication  and  coordination  with  humans  and  simulated  entities,  maintenance  of 
situational  awareness,  and  the  ability  to  accept  and  respond  to  new  orders  while  in  flight.  It  has 
been  successfully  applied  in  a  1997  Department  of  Defense  Synthetic  Theater  of  War  simulation. 
One  of  its  developers  (Jones,  personal  communication,  1999)  sounded  optimistic  about  the 
feasibility  of  a  civil  application  such  as  realistic  radio  communications  simulation. 


4.4.1  Component  Agent  Approach 

Expert  system  development  for  large-scale  computer  simulations  of  environments,  such  as 
tactical  air  combat,  is  an  expensive  and  time-consuming  effort.  Much  of  the  development  of 
TacAir-Soar  and  its  predecessor,  TacBrawler,  took  many  years  to  develop  because  of  the 
extraordinarily  complex  and  dynamic  environment  that  these  systems  are  attempting  to  simulate. 
The  development  of  this  level  of  sophistication  within  a  training  simulation  environment  may  not 
be  cost  effective. 

An  alternative  approach  to  the  application  of  intelligent  systems  which  does  not  require  the 
extensive  development  of  war  gaming  simulations  is  to  develop  a  component  approach  to  the 
application  of  intelligent  systems.  In  this  approach,  smaller,  modular  components  (agents)  are 
added  into  existing  simulation  applications  and  are  programmed  to  provide  expertise  within  a 
narrow  range  of  parameters.  Modularized  agent  technology  could  be  developed  and  applied  to 
emulate  controller  behavior  within  a  given  facility  or  sector  or  even  within  the  confines  of 
particular  approach  procedure.  Each  such  agent  is  provided  with  only  those  rules  necessary  to 
provide  the  simulation  environment  with  the  needed  communications  in  a  timely  fashion. 
Because  of  their  small  size,  such  agents  could  be  readily  adapted  to  a  variety  of  simulation 
environments  without  a  major  re-design  of  the  simulator’s  software. 

This  modular  approach  to  intelligent  agent  development  and  application  has  become  a  cost- 
effective  alternative  to  large-scale  intelligent  systems  development,  particularly  when  agents  are 
intended  as  add-ons  to  existing  software  applications  (Riecken,  1994). 
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4.5 


SUPPORTING  TECHNOLOGIES 


4.5.1  Rapidly  Reconfigurable  Line-Oriented  Evaluation  Scenario  Generator  (RRLOE) 

Technologies  that  may  support  the  automated  provision  of  realistic  radio  communications  during 
simulator  training  events  also  have  been  examined.  The  most  promising  of  these  for  a  script- 
based  approach  is  the  rapidly  reconfigurable  line-oriented  evaluation  scenario  generator 
(Bowers,  Jentsch,  Baker,  Prince,  &  Salas,  1997).  LOE  scenarios  currently  are  developed  and 
tested  by  the  training  organization  and  individually  approved  by  the  FAA.  Training 
organizations  can  only  produce  a  limited  number  of  scenarios,  which  are  valid  for  a  limited  time. 
The  risk  that  the  content  of  the  scenarios  will  become  known  among  pilots  is  high.  RRLOE  will 
use  LOE  event  sets  (modules)  which,  when  FAA  approved,  can  be  assembled  by  the  training 
organization  into  complete  and  valid  LOE  scenarios.  In  addition,  the  RRLOE  scenario  generator 
will  provide  the  instructor  with  either  an  explicit  script  of  ATC  transmissions  or  the  necessary 
information  for  all  communications  directed  from  ATC  to  the  crew. 


4.5.2  Speech  Generation  and  Recognition  Technology 

Any  successful  automated  radio  communications  simulation  system  would  have  to  rely  on 
speech  generation  and  recognition  systems.  Only  a  summary  of  this  technology  and  its 
application  to  flight  simulation  can  be  provided  here.  Significant  progress  in  speech  research 
over  the  past  two  decades  permits  the  generation  of  high-quality  speech  regardless  of  whether 
the  speech  is  synthesized  “from  scratch”  (articulatory  or  formant  synthesis,  e.g.,  DecTalk)  or 
concatenated  from  prerecorded  and  digitized  natural  speech  (concatenative  speech).  In  assessing 
the  state  of  speech  recognition  technology  with  respect  to  its  practical  utility  for  airline  pilot 
training  and  evaluation,  several  criteria  need  to  be  considered.  First,  the  intelligibility  of  voice 
displays  used  in  simulations  needs  to  be  provided  at  a  level  comparable  to  that  of  human  speech 
to  avoid  introducing  additional  cognitive  loads  to  trainees  during  either  training  or  evaluation, 
especially  in  the  absence  of  disambiguating  semantic  context.  Human-speech  like  intelligibility 
is  well  within  the  capabilities  of  existing  higher  quality  speech  systems,  at  least  in  a  quiet 
environment.  In  a  noisy  environment  such  as  the  cockpit,  however,  articulatory  or  formant 
synthesis  may  not  yet  be  able  to  replicate  the  full  richness  of  natural  human  speech  with  all  its 
redundancy,  which  may  be  needed  when  some  of  the  distinguishing  properties  of  individual 
speech  segments  are  masked  by  the  ambient  noise.  Concatenative  synthesis,  which  consists  of 
natural  speech  segments,  should  maintain  some  of  this  redundancy  and  receive  higher 
intelligibility  ratings  in  noise. 

A  second  consideration  is  how  natural  the  synthetic  voice  sounds,  which,  for  lack  of  an  objective 
standard,  is  determined  by  subjective  assessments  by  a  sample  of  end  users.  Naturalness  is  a 
function  of  the  degree  to  which  both  coarticulation  and  prosodic  effects  are  captured. 
Coarticulation  results  in  the  “sound  signature”  of  words  excised  from  a  sentence  looking  quite 
different  from  words  spoken  in  isolation,  while  prosody  mainly  refers  to  pitch,  timing,  and 
intensity.  Currently,  only  concatenative  speech  using  units  large  enough  to  cover  both 
coarticulation  and  prosodic  effects  meet  extremely  high  quality  criteria,  at  a  high  computer 
memory  cost.  Theoretical  research  in  prosody,  in  particular  with  respect  to  pitch  and  timing, 
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however,  is  progressing  internationally  and  its  results  are  being  implemented  by  the  industry. 
This  will  greatly  improve  user  acceptance  of  synthetic  speech,  which  is  important  to  avoid  pilot 
stress  and  distraction  especially  for  advanced  training  and  evaluation.  There  is  no  reason  to  rule 
out  machine  speech  generation  for  future  simulator  applications. 

The  main  stumbling  block,  however,  may  be  automatic  speech  recognition,  although  recognition 
rates  have  been  dramatically  improved  over  the  past  5  years.  The  main  problems  that  need  to  be 
overcome  by  an  automatic  speech  recognizer  to  be  successful  are  coarticulation  (words  can 
occur  in  an  unlimited  number  of  contexts),  and  variability  between  and  sometimes  even  within 
speakers  in  articulating  the  same  word.  The  first  problem  may  be  less  prominent  in  air  traffic 
control,  which  uses  a  restricted  and  highly  regulated  phraseology.  The  second  problem  refers  to 
the  fact  that  each  speaker  will  articulate  the  same  word  or  phrase  slightly  differently.  This  may 
be  due  to  physical  differences  in  the  vocal  tract,  such  as  found  between  men  and  women,  or 
different  linguistic  experiences,  such  as  geographic  or  social  accents.  Differences  in  articulation 
can  be  pronounced  even  within  the  same  speaker,  due  to  variations  in  speaking  rate,  a  cold,  or 
emotional  state.  Even  with  a  highly  regulated  phraseology  such  as  used  in  aviation,  extensive 
“training”  of  the  system  with  a  large  set  of  utterances  of  each  speaker  will  be  required  to  achieve 
satisfactory  reliability. 

With  regard  to  speaking  rate,  speech  recognition  systems  can  now  recognize  rates  near  those  of 
normal,  conversational  speech  (e.g.,  about  180  words  per  minute),  but  no  system  has  yet 
achieved  the  much  higher  speech  rates  often  found  in  aviation  operations  (e.g.,  more  than  250 
words  per  minute,  see  Lee,  1999).  This  suggests  that  simulator  applications  of  this  technology 
may  have  to  await  further  refinements.  Additionally,  the  speaker  training  required  of  these 
systems  to  achieve  very  high  levels  of  recognition  accuracy  may  prove  problematic  for  training 
management.  However,  the  implementation  of  human  speech  perception  and  production  research 
findings  as  well  as  advanced  mathematical  tools  make  it  conceivable  that  these  limitations  may 
be  overcome  in  the  future.  For  this  reason,  the  application  of  speech  recognition  technology  in 
communications  simulation,  necessary  for  a  fully  automated  communications  simulation,  should 
be  considered  as  a  potentially  long-term  enabling  technology  for  future  development  in 
communications  simulation. 
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5.  CONCLUSIONS 


The  findings  documented  in  this  report  are  as  follows: 

■  The  literature  emphasizes  the  need  for  a  realistic  radio  communications  environment  as  a  key 
element  in  training  and  evaluation  scenarios.  Not  only  does  it  identify  radio  communications 
as  a  contributor  to  many  incidents  and  accidents,  but  it  also  underscores  its  importance  for 
effective  CRM  (where  ATC  and  company  are  part  of  the  crew)  and  task  management 
training. 

■  Subject  matter  experts  perceive  realistic  radio  communications  as  an  important  element  in 
pilot  simulator  training  and  evaluation,  enhancing  pilots’  distraction  management  and 
situation  awareness  skills  while  breaking  the  “simulator  mindset.” 

■  Airlines  mostly  leave  the  simulation  of  radio  communications  to  the  individual  I/Es,  thus 
increasing  I/E  workload  while  reducing  pilot  workload  because  the  already  taxed  I/Es  cannot 
provide  a  realistic  communications  load.  This  impoverished  simulation  environment  may 
compromise  the  transfer  of  skills  between  the  simulator  and  the  airplane. 

■  The  technology  to  simulate  ATC/Company  and  partyline  automatically  is  still  immature, 
although  there  is  promising  progress  in  intelligent  systems  and  automated  speech 
generation/recognition. 

The  underlying  premise  of  this  work — that  the  provision  of  realistic  radio  communications 
during  training  and  evaluation  of  airline  pilots  would  enhance  safety— has  been  confirmed  thus 
far.  Airlines,  however,  are  unlikely  to  allocate  the  necessary  funds  for  provision  of  a  full  radio 
communications  environment  during  simulator  training  and  evaluation  unless  there  is  a  proven 
gain  in  safety  and  training  effectiveness  and  a  requirement  from  the  FAA.  The  technology 
development  seems  to  be  largely  market  driven,  but  would  of  course  be  furthered  by  such  a 
requirement.  Therefore,  a  proof-of-concept  is  required  that  will  empirically  demonstrate  the 
benefit  of  realistic  radio  communications  for  safety  of  operations.  A  first  step  may  be  to 
systematically  compare  the  communications  load  and  its  effect  on  pilot  workload  and  CRM 
behaviors  during  revenue  operations  in  the  air  versus  LOFT/LOEs.  Another  step  may  be  an 
assessment  of  pilots  who  have  just  completed  initial  qualification  in  their  first  fleet.  The  question 
of  interest  would  be  how  prepared  such  pilots  are  to  handle  ATC  and  company  communications 
requirements  during  their  first  Initial  Operating  Experience  (IOE)  flights.  This  may  be  followed 
up  by  a  carefully  conceived  study  examining  the  effects  of  different  levels  and  approaches  to 
achieve  radio  communication  realism  on  transfer  of  training  acquired  in  the  simulator  to  the 
airplane  or  at  least  the  simulator  with  realistic  radio  communications  as  a  stand-in  for  the 
airplane.  Full  skill  transfer  to  and  from  the  airplane  is  a  critical  issue,  if  simulator  use  for  training 
and  evaluation  is  to  be  mandated. 
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APPENDIX  A 


RESULTS  OF  THE  APPROACH  AND  LANDING  ACCIDENT 
REDUCTION  TASK  FORCE  REPORT  AND  RELATED  STUDIES 


The  importance  of  efficient  interactions  with  ATC  and  company  (Khatwa  &  Helmreich,  1999) 
are  pointed  out  in  the  report  of  the  Approach  and  Landing  Accident  (ALA)  reduction  task  force 
of  the  Flight  Safety  Foundation.  The  task  force  based  its  conclusions  on  an  analysis  of  76  ALAs 
and  serious  incidents  involving  jet  and  turboprop  aircraft  and  occurring  between  1984  and  1997 
(inclusive).  “Incorrect  or  inadequate  ATC  instruction/advice/service”  was  a  causal  factor  in  33 
percent  of  all  occurrences.  It  ranked  eleventh  among  the  most  common  causal  factors  after  “Poor 
professional  judgment/airmanship”  in  first,  “Lack  of  positional  awareness”  in  fourth,  and  long 
before  “Interaction  with  automation”  in  seventeenth  place.  “[Demanding  ATC  clearances”  are 
also  explicitly  mentioned  in  context  with  even  higher  ranking  causal  factors  such  as  the  sixth- 
placed  “Flight-handling  difficulties,”  the  eighth-placed  “Press-on-itis,”  or  the  tenth-placed  “Slow 
and/or  low  on  approach.”  In  many  cases  of  “press-on-itis,”  “a  breakdown  in  CRM  between  the 
flight  crew  and  ATC”  was  observed. 

Incorrect  or  inadequate  ATC  instruction/advice/service  may  include  incorrect  radar  vectoring, 
incorrect  (or  absence  of)  essential  traffic  information,  or  inadequate  controller  technique  for 
dealing  with  aircraft  facing  difficulties.  Late  runway  assignment  notifications  or  last-minute 
changes  may  cause  rushed  approaches.  ATC  may  be  unaware  of  the  demands  and  limitations  of 
the  glass  cockpit,  where  such  clearances  may  require  reprogramming  of  the  flight  management 
system  (FMS).  This  greatly  increases  workload,  undermines  CRM  and  reduces  situational 
awareness.  Potential  results  are  unstable  conditions,  overruns,  CFIT  (Controlled  Flight  Into 
Terrain),  or  loss  of  control.  Several  of  the  problems  mentioned  in  this  paragraph  were  recognized 
as  potentially  contributing  factors  to  the  1995  American  Airlines  Flight  965,  Boeing  757  CFIT  in 
Cali,  Columbia,  which  cost  159  lives  (Simmon,  1998). 

Other  ATC-communications  related  causal  factors  in  the  report  that  are  not  included  in  the  33 
percent  are  misunderstood  or  missed  communications  such  as  missed  readbacks,  call-sign 
confusions,  and  simultaneous  transmissions  (12  percent  of  occurrences).  Instances  of  controllers 
and  crews  using  non-standard  phraseology  are  also  mentioned.  Communications  can  become 
especially  difficult  when  non-native  English  speakers  and  listeners  are  involved,  as  the  1990 
Avianca  Airlines  Boeing  707-32 IB  crash  over  Long  Island  demonstrates.  In  this  accident,  the 
Colombian  crew  was  unsuccessful  in  communicating  a  fuel  emergency  to  the  American  ATC 
(NTSB,  1991).  In  addition  to  the  ALA  survey,  Khatwa  and  Helmreich  mention  an  ongoing  study 
including  line  audits  of  102  flight  segments  to  and  from  destinations  with  non-native  English 
speaking  ATC  (Klinect,  Wilhelm,  &  Helmreich,  1999).  One  of  the  most  frequent  errors  observed 
involved  crew  interaction  with  ATC.  Language  problems  may  also  have  exacerbated  the 
communications  problems  encountered  by  the  American  Airlines  crew  over  Cali  (see  above). 

Both  the  Avianca  and  the  American  Airlines  crashes  show  that  efficient  communication  between 
flight  crew  and  ATC/company  is  especially  important  during  emergency  situations.  The  ALA 
reduction  task  force  relates  several  instances  where  ambiguous  emergency  messages  from  the 
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crew  were  not  cleared  up  by  ATC  follow-up,  a  factor  that  is  also  mentioned  in  the  Cali  accident 
(Simmon,  1998).  In  other  cases,  ATC  may  have  contributed  to  confusion  and  distraction  of  the 
crew,  e.g.,  with  unnecessary  requests  for  information  especially  during  the  “sterile  cockpit” 
period  below  10,000  feet  (Sumwalt,  1993).  The  ALA  task  force  also  reports  instances  where 
crews  repeatedly  ignored  urgent  warnings  from  ATC. 

The  task  force  points  out  that  such  examples  demonstrate  poor  CRM  between  flight  crews  and 
ATC  and  recommend  that  regulatory  authorities  should  “[rjequire  operators  to  provide  all 
operational  personnel  (e.g.,  flight  crew,  cabin  crew,  air  traffic  controllers)  effective  training  in 
CRM.”  It  also  recommends  that  operators  “[ijnclude  training  scenarios  that  allow  crews  to 
experience  overload,  task  saturation,  loss  of  situational  awareness,  out-of-control  and  too-far- 
behind-the-aircraft  situation,  and  communications  in  stressful  circumstances.”  Joint  training 
sessions  should  be  held  between  pilots  and  air  traffic  controllers  including  scenarios  that 
“promote  mutual  understanding  of  issues  on  both  the  flight  deck  and  in  the  ATC  environment, 
and  foster  improved  communications  during  emergency  situations.”  The  use  of  standard  ICAO 
phraseology  should  be  emphasized  (see  also  Jones  and  Tesmer,  1999,  for  training 
recommendations) . 

In  summary,  realistic  radio  communications  in  carefully  constructed  simulator  training  and 
evaluation  scenarios  give  pilots  the  opportunity  to  practice  how  to  recognize  and  deal  with 
incorrect,  inadequate,  or  too  demanding  ATC  instructions.  They  also  allow  pilots  to  practice 
correct  phraseology,  especially  during  emergencies  and/or  with  non-native  English-speaking 
ATC. 
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APPENDIX  B 


REALISTIC  RADIO  COMMUNICATIONS  FOR  TASK  MANAGEMENT 

TRAINING 


THE  NEED  FOR  TASK  MANAGEMENT  TRAINING 

In  an  intuitive  summary  on  the  cognitive  underpinnings  of  task  management  based  on  Norman 
and  Shallice  (1986;  see  also  Shiffrin  &  Schneider,  1997),  Dismukes  et  al.  explain  that  humans 
have  two  cognitive  systems  with  which  they  perform  tasks,  one  involving  conscious  control,  the 
other  operating  in  an  automatic  fashion  (Dismukes,  Young,  and  Sumwalt,  1998).  The  conscious 
system  operates  slowly  and  methodically,  performing  one  operation  at  a  time.  Learning  a  new 
skill  typically  requires  conscious  processing.  Dismukes  et  al.  explain  that  “this  is  why  learning  to 
drive  a  car  or  flying  an  airplane  at  first  seems  overwhelming:  the  multiple  demands  of  the  task 
exceed  conscious  capacity.”  As  a  learner  of  a  skill  progresses,  the  repetitive  aspects  of  a  task  will 
become  automatic,  requiring  little  or  no  conscious  effort,  whereas  novel  aspects  continue  to 
require  conscious  attention.  Since  any  human  language  allows  the  generation  of  infinitely  many 
novel  sentences  (see,  e.g.,  Chomsky  1965),  communication  is  one  activity  that  allows  little 
automatic  processing,  even  if  it  is  somewhat  regulated  as  are  communications  between  pilots  and 
ATC.  Aside  from  novelty,  other  attributes  of  a  task  that  require  conscious  control  listed  by 
Dismukes  et  al.  are  difficulty,  criticality,  or  danger.  Finally,  the  presence  of  competing  activities 
(presumably  of  equal  urgency)  and  “when  an  automatic  process  must  be  overridden  to  prevent 
habit  capture  (such  as  when  trying  to  catch  a  turnoff  when  taking  a  different  route  home  from 
work)  require  a  shift  from  automatic  to  conscious  control. 


Given  that  conversations  have  the  power  to  capture  all  our  conscious  capacity,  it  is  not  surprising 
to  learn  that  radio  communications  have  been  found  to  be  a  close  second  contributor  to 
monitoring  errors  (Sumwalt,  Morrison,  Watson,  and  Taube,  1997).  In  their  study,  Sumwalt  et  al. 
reviewed  200  of  800  ASRS  reports  on  monitoring  errors  submitted  between  1992  and  1996. 
Monitoring  errors  were  defined  as  a  “failure  to  adequately  watch,  observe,  keep  track  of,  or  cross 
check”  the  aircraft’s  trajectory  or  systems.  Safety  consequences  ranged  from  taxi  or  flight  path 
deviations  and  runway  incursions  to  speed  violations  to  system  or  equipment  damage  or 
shutdown  to  loss  of  control.  For  the  170  reports  that  indicated  the  tasks  or  functions 
accomplished  shortly  before  or  during  the  error,  45  percent  reported  being  involved  with  cockpit 
automation/navigation  systems  and  42  percent  with  radio  communications  (ATC,  company,  or 
obtaining  ATIS).  Cockpit  documentation  (checklists,  etc.)  was  in  third  place  with  39  percent 
(some  reporters  listed  several  activities).  The  majority  of  errors  occurred  during  climb,  followed 
by  descent,  descent  transition,  and  approach. 

Similar  findings  are  reported  by  McElhatton  et  al.  who  observed  that  crossing  restriction  altitude 
deviations  on  Standard  Instrument  Departures  (SIDs)  and  Terminal  Arrival  Routes  (STARs)  tend 
to  occur  most  frequently  during  descent  (77  versus  33  percent  of  172  ASRS  reports  studied, 
McElhatton,  Buchanan,  and  Drew,  1998).  McElhatton  et  al.  hypothesize  that  the  preponderance 
of  deviations  during  descent  can  be  traced  back  to  increased  workload  due  to,  among  others, 
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ATIS  and  company  communications.  Cockpit  workload  was  indeed  cited  as  a  factor  in  44 
percent  of  the  reports.  Again,  “high  quantity  radio  communications  with  ATC”  were  the  second 
most  frequently  mentioned  workload  issue  with  just  one  less  citation  than  “FMS  programming,” 
and  tied  with  “lack  of  planning  on  the  part  of  the  flight  crew  that  led  to  time  compressions  (such 
as  cabin  attendant  in  cockpit).”  In  fact,  overall  radio  communications  issues  may  have  even 
surpassed  automation  issues,  given  that  company  communications  were  included  in  “Other”  in 
third  position. 

One  might  object  that  ASRS  reports  represent  a  biased  sample  due  to  their  voluntary  nature.  The 
fact  that  task  management  and  related  flight  path  errors  increase  with  the  number  of  concurrent 
tasks,  however,  has  been  confirmed  in  controlled  experiments  as  well.  Chou  et  al.  investigated 
the  significance  and  nature  of  the  task  management  process  by  first  conducting  a  review  of 
NTSB  accident  and  ASRS  incident  reports  and  then  testing  their  findings  in  a  controlled 
simulator  experiment  (Chou,  Madhavan,  and  Funk,  1996).  They  describe  cockpit  task 
management  (CTM)  as  consisting  of  the  following  seven  functions:  1)  task  initiation  when 
appropriate  conditions  exist;  2)  task  monitoring;  3)  task  prioritization;  4)  resource  allocation;  5) 
task  interruption  when  resources  need  to  be  allocated  to  a  higher  priority  task;  6)  task  resumption 
when  priorities  change  or  resources  become  available;  and  7)  task  termination  when  task  has 
been  completed,  cannot  be  completed,  or  has  become  irrelevant.  CTM  errors  are  classified  to 
correspond  to  the  aforementioned  CTM  functions. 

In  their  accident  report  study,  they  found  CTM  errors  in  23  percent  of  the  324  accidents 
reviewed.  This  number  rose  to  49  percent  of  470  reports  reviewed  for  the  incident  study.  In  both 
studies,  task  initiation  errors,  such  as  early  descents,  late  configurations,  and  failures  to  tune 
navigation  and  communication  radios,  were  in  the  lead.  Task  prioritization  errors  including 
distractions  by  weather  and  traffic  watches  were  a  close  second,  especially  for  the  incidents.  Last 
were  termination  errors  (such  as  early  autopilot  disengagements,  altitude  overshoots,  and 
improperly  continued  landings  under  unsafe  conditions). 

The  objective  of  the  simulator  study  was  to  elicit  the  CTM  errors  found  in  the  previous  analyses 
and  to  identify  the  contributing  factors.  Mainly  non-pilot  participants  flew  a  low-fidelity 
simulator  after  training,  in  one  of  six  scenarios  of  varying  degrees  of  difficulty.  Degree  of 
difficulty  was  defined  as  a  function  of  resource  requirements,  number  of  concurrent  tasks,  and 
flight  path  complexity.  Performance  was  measured  by  assessing  response  times  to  system  faults, 
flight  path  errors,  task  prioritization,  and  late  task  initiation.  Average  response  time  and  late  task 
initiation  were  significantly  increased  only  by  visual,  manual,  and  mental  resource  requirements, 
but  not  by  the  number  of  concurrent  tasks  or  flight  path  complexity.  The  only  flight  deviation 
measures  that  were  affected  by  any  of  the  scenario  manipulations  were  heading,  where 
deviations  increased  by  the  combination  of  flight  path  complexity  and  the  number  of  tasks,  and 
altitude  deviations,  which  were  increased  by  increased  mental  resource  requirements.  Task 
prioritization,  however,  was  greatly  degraded  by  both  resource  requirements  and  the 
combination  of  flight  path  complexity  and  number  of  concurrent  tasks. 

One  last  study  to  be  mentioned  here  that  specifically  investigated  interruption  management 
experimentally  used  ATC  clearances  to  interrupt  commercial  airline  pilots  as  they  performed 
procedures  during  approaches  in  the  simulator  (Latorella,  1996).  Latorella  found  that  ATC 
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interruptions  significantly  increased  procedure  performance  errors  as  well  as  flightpath 
management  activity  (number  of  attitude  lateral  control  inputs). 

THE  METHOD  FOR  TASK  MANAGEMENT  TRAINING 

Now  that  it  has  been  established,  via  both  anecdotal  and  empirically  evidence,  that  task 
management  skills  need  to  be  trained  and  evaluated,  how  can  this  best  be  accomplished? 

\ 

Simulator  Task  Management  Training  Can  Transfer  to  In-Flight  Performance/Behavior 

A  first  finding  is  that  task  management  trained  even  in  a  simulation  with  low  physical  fidelity 
may  indeed  transfer  to  flight  (Gopher,  Weil,  and  Bareket,  1994).  The  training  device  used  by 
Gopher  et  al.  consisted  of  an  extremely  complex  computer  game  requiring  subjects  to  control  the 
movement  of  a  spaceship  while  aiming  and  firing  missiles  to  destroy  a  space  fortress.11  Limited 
resources,  constant  enemy  attacks,  and  the  requirement  to  verbalize  one’s  own  actions  ensured 
task  difficulty.  Similarity  of  task  demands  with  real  flight  was  seen  in  the  shared  requirement  for 
continuous  and  discrete  manual  control,  visual  and  spatial  orientation,  procedural  knowledge 
involving  long-  and  short-term  memory,  high  attention  demands  under  severe  time  constraints, 
and  the  presence  of  in-flight  communication  (verbalizations).  Gopher  et  al.  compared  actual 
flight  performance  of  two  groups  trained  with  different  strategies  (to  be  discussed  below)  in  the 
space-fortress  game  with  a  control  group  without  training  and  found  a  clear  advantage  of  the 
space-fortress  trained  groups  over  a  control  group  of  matched  ability.  Gopher  et  al.  make  a  point 
of  explaining  that  it  is  unlikely  that  these  results  could  be  due  to  the  so-called  Hawthorne  effect, 
which  purports  that  any  group  receiving  special  treatment  over  what  another  group  receives 
should  show  an  advantage.  All  subjects  in  this  study  were  highly  motivated  volunteers  from  an 
exclusive  military  flight  school,  but  participation  in  the  study  was  only  a  small  and  unimportant 
fraction  of  their  daily  activities  that  was  not  welcomed  with  enthusiasm.  The  space-fortress 
training  was  regarded  as  an  extra  burden,  not  a  privilege.  A  cover  story  was  aimed  at  eliminating 
further  bias.  Participation  or  lack  of  such  in  the  space-fortress  training  was  not  evident  to  the 
flight  performance  evaluators.  Interestingly,  the  computer  game  was  perceived  as  such  a 
successful  training  tool  that  it  was  later  incorporated  into  the  regular  curriculum  of  the  Israeli  Air 
Force. 

Whole-Task  Versus  Part-Task  Training 

The  1994  Gopher  et  al.  study  also  served  to  compare  the  effects  of  two  different  training 
strategies  for  the  two  space-fortress  game  groups.  The  so-called  full-training  group  was  first 
trained  with  a  sophisticated  hierarchical  part-task  technique  of  working  its  way  up  through  a 
series  of  simplified  “part-task”  games  of  increasing  complexity  before  the  integrated  game  with 
its  full  task  load  was  presented,  all  the  while  receiving  verbal  tips  and  individual  coaching.  This 
technique  has  been  developed  by  Frederiksen  and  White  and  had  been  shown  to  lead  to  superior 
performance  than  simple  “whole-task”  training,  at  least  in  the  context  of  the  space-fortress  game 
or  related  contexts  (1989).  Although  some  effort  was  taken  to  test  transfer  to  a  different  context, 
subjects  still  had  to  manipulate  an,  albeit  somewhat  differently  behaving,  space  ship  on  a  screen. 

11  All  the  studies  using  the  space  fortress  game  are  part  of  a  concerted  effort  funded  by  a  DARPA  contract  to  the 
University  of  Illinois,  principal  investigator  Emanuel  Donchin. 
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No  transfer  to  a  real-life  task  requiring  the  trained  skills  was  attempted.  In  addition  to  the 
hierarchical  part-task  training,  the  full  training  group  in  the  Gopher  et  al.  1994  study  received  the 
emphasis-change  training  described  below. 

The  second  space-fortress  game  group,  dubbed  “emphasis-only”  group,  was  trained  under  an 
emphasis-change  regime  developed  by  Gopher  and  colleagues  specifically  for  complex  skill 
acquisition  in  an  earlier  study  (Gopher,  Weil,  and  Siegel,  1989).  This  group  practiced  the  whole 
game  with  its  full  load  at  all  times,  but  was  “led  through  instructions  and  auxiliary  feedback 
indicators  to  vary  their  focus  of  attention  in  different  game  trials  on  different  aspects  of  the 
game.”  In  the  earlier  study,  this  technique  had  proven  superior  to  exposure  to  the  whole  task 
without  instruction,  but  only  in  the  context  of  the  space  fortress  game.  No  transfer  to  real-life 
tasks  was  tested  in  this  earlier  work. 

As  is  known  from  the  previous  section,  both  game  groups  outperformed  the  no-game  group  in 
the  flights  of  the  1994  study,  showing  high  transfer  of  training  for  the  acquired  attention 
management  strategies.  Moreover,  despite  the  fact  that  the  part-task  trained  group  achieved 
considerably  higher  game  scores  than  the  emphasis-change  group,  the  two  groups  did  not  differ 
in  subsequent  flight  performance.  This  may  indicate  that  the  whole-task  training  with  emphasis- 
change  experienced  by  both  groups  “promoted  the  development  of  skill  components  that  could 
be  transferred  and  generalized  to  flight,  whereas  the  part-task  training  and  verbal  tips  contributed 
only  to  skill  elements  that  were  exclusively  relevant  to  space  fortress  performance.” 

This  interpretation  is  supported  by  work  by  Fabiani  et  al.  who  trained  three  space-fortress  game 
groups  with  the  whole  task,  either  with  or  without  emphasis  training,  or  the  hierarchical  part-task 
training  method  (Fabiani,  Buckley,  Gratton,  Coles,  Donchin  and  Logie,  1989).  Again,  the  part- 
task  group  performed  considerably  better  than  the  emphasis-change  group  in  the  space-fortress 
game  after  training.  However,  when  a  second  unrelated  task,  such  as  rapid  foot  tapping,  was 
introduced  in  addition  to  the  space-fortress  tasks,  the  emphasis-change  group  outperformed  the 
part-task  group.  In  all  conditions,  the  whole-task  control  group  lagged  behind  the  two  other 
groups. 

In  general,  careful  whole-task  practice  may  thus  generalize  and  transfer  better  than  part-task 
practice  to  novel  situations,  especially  if  they  require  multi-tasking  skills  such  as  flying  an 
airplane.  This  was  also  shown,  for  a  non-aviation  multi-task  application,  by  Detweiler  and  Lundy 
(1995).  They  compared  the  effects  of  single  and  dual  task  training  on  dual  task  performance  in 
visual  search  tasks  involving  word-category  and  pattern  matching.  During  the  eight  training 
blocks,  the  learning  curve  was  steeper  for  the  single  task  group,  although  this  advantage  tended 
to  diminish  during  the  course  of  training  at  least  for  the  most  important  variable  of  hits.  When 
transferring  to  the  dual  task  condition  after  training,  however,  although  the  single  task  trained 
group  did  show  some  effect  of  prior  training,  the  dual  task  trained  group  showed  a  substantial 
performance  advantage  over  the  single  task  group.  This  advantage  was  maintained  over  at  least 
four  training  blocks,  although  the  single  task  trained  group  showed  a  trend  for  catching  up  later. 
Detweiler  and  Lundy  attribute  this  advantage  to  an  opportunity  of  1)  developing  strategies  for 
focusing  attention  on  two  tasks  at  once;  2)  acquiring  skills  in  performing  both  type  of  memory 
comparisons  together;  and  3)  learning  how  to  select  and  execute  different  responses  in  close 
sequence. 
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In  aviation,  the  motivation  for  part  training  is  often  to  complete  part  of  the  training  in  a  low-cost 
part-task  simulation  and  thus  saving  time  in  the  costly  high-fidelity  device.  This  is  often 
supported  by  the  intuitively  attractive  reasoning  that  at  least  initial  training  may  be  enhanced  by 
sparing  trainees  from  being  exposed  to  too  many  tasks  all  at  once.  This  perception  was  shared  by 
one  of  the  participants  of  the  instructor/evaluator  survey  who  felt  that  “realistic  radio 
communications  are  more  important  for  LOE  and  LOFT  than  for  initial  training.”  The  literature, 
however,  does  not  appear  to  support  these  intuitions  for  aviation,  where  unsuccessful  multi¬ 
tasking  has  led  to  tragic  accidents  such  as  the  1978  DC-8  crash  into  a  Portland  suburb,  when  a 
crew  distracted  by  a  landing  gear  problem  ran  out  of  fuel  (Diehl,  1992). 
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APPENDIX  C 

INSTRUCTOR/EVALUATOR  QUESTIONNAIRE 

Realistic  Radio  Communications  Simulation 

Questionnaire 


First  Name: . 

Last  Name: . 

Title: . 

Company: . 

Please  mail  the  completed  questionnaire  to: 

Dr.  Alfred  T.  Lee 
Beta  Research,  Inc. 

P.O.  Box  66423 
Scotts  Valley,  CA  95067 


Thank  you  very  much  for  participating  in  our  study.  Your  input  is  critical  for  evaluating 
the  effect  of  current  radio  communications  simulation  practices  on  both  instructor 
workload  and  training/evaluation  efficiency;  and  to  determine  the  need  for  automated 
tools  to  provide  a  realistic  radio  communications  environment. 

Questions  or  comments?  Contact  Dr.  Lee  at  a fleets*  ix. netcom. com  or  408-353-2665 
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1)  Types  of  Communications  Simulated: 

First,  we  would  like  to  know  which  types  of  communications  are  being  simulated  in  your 
company’s  simulator  training  and  evaluation  events  (LOFT,  LOE,  and  SPOT).  Note:  At 
this  point,  we  are  only  asking  about  communications  with  own  aircraft. 

In  the  tables  below,  check  the  boxes  to  indicate  which  types  of  communications  are 
incorporated  into  your  company’s  Line  Operational  Simulation  (LOS). 


Communications  with  ATC 

5^ 

w1d 

ATIS 

Clearance 

Delivery 

Tower 

Ground 

Tower 

Local 

Approach/ 

Departure 

Center 

mm 

□ 

□ 

□ 

□  i 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

Communications  with  Company 

1  :  •.  •  .  • 

Type  of 

LOS 

j;  ■li-'j.l;;:'.1  v  . 

Dispatch 

Ramp/Gate 

Cabin 

Other: . 

(Please  indicate,  e.g.,  maintenance, 
customer  service) 

!§OFT  - 

□ 

□ 

□ 

□ 

LOE 

□ 

□ 

□ 

□ 

SPOT 

□ 

□ 

□ 

□ 

Comments: 
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2)  How  are  Communications  Simulated? 

We  would  like  to  learn  exactly  howVnese  communication  simulations  are  implemented. 
Please  check  the  boxes  that  apply. 

I/E  provides  -  one  instructor  provides  all  simulator  operation  including  radio 
communications 

Live  “controllers”  -  an  additional  staff  member  focuses  on  radio 
communications  simulation 

Recorded  controller  voice  -  prerecorded,  automated  controller  voice 
Synthetic  controller  voice  -  prerecorded,  automated  synthetic  controller  voice 

If  you  check  “Other”,  please  explain  in  the  comments  section  below. 


Communications  with  ATC 

1111111 

ATIS 

Clearance 

Delivery 

Tower 

Ground 

Tower 

Local 

Approach/ 

Departure 

Center 

□ 

□ 

□ 

□ 

□ 

□ 

|,  !S  «f  '  r  n 

□ 

□ 

□ 

□ 

□ 

□ 

iwSIBlIiiiiil 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

MsipispatsiiKpiii 

□ 

□ 

□ 

□ 

□ 

□ 

Comments: 


Communications  with  Company 

Implementation 

Dispatch 

Ramp/Gate 

Cabin 

Other: . 

(Please  indicate,  e.g.,  maintenance, 
customer  service) 

□ 

□ 

□ 

lE 

□ 

C 

lE 

□ 

1  if!  ||g|| ..  *  H  *  | 

□ 

□ 

□ 

□ 

1  co9^oll®r 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

Comments: 


55 


3)  Effect  and  Overall  Importance  of  Radio  Communications  Simulations: 

Here  we  would  like  your  opinion  on  how  your  company’s  radio  communication 
simulation  practices  impact  instructor  workload  compared  to  instructing/evaluating  pilots 
in  the  air  with  real  radio  communications  (to  own  airplane).1 

We  also  would  like  your  opinion  on  how  those  practices  impact  pilot  workload,  again 
compared  to  what  the  pilot  experiences  in  the  air. 

Finally,  we  would  like  you  to  rate  the  overall  importance  of  realistically  simulating  each 
type  of  communication  on  training/evaluation  effectiveness. 


Please  circle  the  ratings  that  apply. 


Communications  with  ATC 

'  rnmnaroH 

|$!  !■ :  •• :  A  V 

ATIS 

Clearance 

Delivery 

Tower 

Ground 

|  instructor  workload 

s,n’ta"°n  * 

1  2 
Lower 
than  a/c 

3  4 
Same 

5 

Higher 

1  2 
Lower 

3  4 
Same 

5 

Higher 

1  2 
Lower 

3  4 
Same 

5 

Higher 

.  Pilot  workload  in 
-simulation  is: 

1  2 
Lower 

3  4 
Same 

5 

Higher 

1  2 
Lower 

3  4 
Same 

5 

Higher 

1  2 
Lower 

3  4 
Same 

5 

Higher 

Training  j  ^ 

effectiveness  in 
^simulation  is: 

,  v:-:' 

s?'?! i ( l.f ! ;.'1 ' ;■  "v'd  - r  •  r-  •  1 ' . 1  !-  • 

1  2 
Lower 

3  4 
Same 

5 

Higher 

1  2 
Lower 

3  4 
Same 

5 

Higher 

1  2 
Lower 

3  4 
Same 

5 

Higher 

:  Overall  importance 
linlsimulation  is: 

1  2 
Very  Low 

3  4  5 

Very  High 

1  2 
Very  Low 

3  4  5 

Very  High 

1  2 
Very  Low 

3  4  5 

Very  High 

Comments: 


1  For  this  comparison,  you  may  have  to  imagine  instructor  workload  with  real  radio  in  the  air. 
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Communications  with  ATC  (cont.) 


tes- 

Tower 

Local 

Approach/ 

Departure 

Center 

1  2  3  4  5 

Lower  Same  Higher 
than  a/c 

1  2  3  4  5 

Lower  Same  Higher 

1  2  3  4  5 

Lower  Same  Higher 

1  2  3  4  5 

Lower  Same  Higher 

1  2  3  4  5 

Lower  Same  Higher 

1  2  3  4  5 

Lower  Same  Higher 

1  2  3  4  5 

Lower  Same  Higher 

simulation  s: 

1  2  3  4  5 

Very  Low  Very  High 

Comments: 


3)  Effect  and  Overall  Importance  of  Communications  Simulations  (cont.): 


Communications  with  Company 

Simulation 

I  Effect  on: 

Dispatch 

Ramp/Gate 

Cabin 

Other . 

(Please  indicate): 

workload  j$r|  1 1  : 
^simulationlslllll 

HSttPl 

1  2 
Lower 
than  a/c 

3  4 
Same 

5 

Higher 

1  2 
Lower 

3  4 
Same 

5 

Higher 

1  2 
Lower 

3  4 
Same 

5 

Higher 

1  2 
Lower 

3  4 
Same 

5 

Higher 

iliiliiPSi 

simulation  is: 

1  2 
Lower 

3  4 
Same 

5 

Higher 

1  2 
Lower 

3  4 
Same 

5 

Higher 

1  2 
Lower 

3  4 
Same 

5 

Higher 

1  2 
Lower 

3  4 
Same 

5 

Higher 

1  2 
Lower 

3  4 
Same 

5 

Higher 

1  2 
Lower 

3  4 
Same 

5 

Higher 

importance  in 
simulation  is: 

1  2  3  4  5 

Very  Low  Very  High 

1  2  3  4  5 

Very  Low  Very  High 

1  2  3  4  5 

Very  Low  Very  High 

Comments: 
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4)  Simulation  of  ATC  Communications  With  Other  Aircraft  (partyline): 

Now,  we  would  like  to  find  out  about  your  company’s  simulations  of  ATC 
communications  with  other  aircraft  (and  ground  vehicles).  In  addition,  we  would  like  to 
know  if  visual  simulations  of  aircraft  and  ground  vehicles  are  incorporated  through  out- 
the-window  views  or  TCAS. 

Please  indicate  below  which  simulations  of  other  traffic  are  implemented  in  your 
company’s  training  and  evaluation  events. 


mmmi 

Type  of 
Simulation 

Out-the-Window  View 

TCAS 

Communications  to 

Other  (clearances) 

Communications  from 

Other  (readbacks) 

Airport  surface 

□  Aircraft 

□  Emerg’cy  Vehicles 

□ 

□ 

I]  Aircraft 

I]  Emerg’cy  Vehicles 

I]  Aircraft 

□  Emerg’cy  Vehicles 

"Terminal  ' 

$]!:  -c .K-;-  .  '■  , 

□ 

□ 

□ 

□ 

En-route  ^::;v 

MPf!*:. 

□ 

□ 

□ 

□ 

Comments: 


5)  How  are  ATC  Communications  to  Other  Aircraft  Simulated? 

(See  Question  2  for  meanings  of  ratings.) 


Communications  with  ATC 

Clearance 

Tower 

Tower 

Approach/ 

Center 

Implementation 

III  .. 

Delivery 

Ground 

Local 

Departure 

Not  simulated 

if: ~  fiif. ffff  '  ■■ 

□ 

□ 

□ 

□ 

□ 

l/E  provides 

□ 

□ 

□ 

□ 

□ 

Live  “controllers” 

□ 

□ 

□ 

□ 

□ 

(Recorded 'Controller 

|  .  1:^1  '  =. 

voice, 

□ 

□ 

□ 

□  1 

□ 

Synthetic  controller 
voice 

□ 

□ 

□ 

□ 

□ 

Other: . . 

■■■■  ' 

□ 

□ 

□ 

□ 

□ 

Comments: 
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6)  Effect  and  Overall  Importance  of  Partyline  Simulation: 

a)  Please  rate  the  effect  of  your  company’s  partyline  simulation  practices  on  instructor 
workload  and  on  pilot  workload,  again  compared  to  instructing/evaluating  or  piloting 
in  the  air  with  a  real  partyline.2 


b)  Whether  or  not  your  company  engages  in  partyline  simulation,  please  rate  whether 
you  think  the  simulation  of  the  partyline  increases  or  decreases  training  and  evaluation 
effectiveness  and  how  important  you  feel  it  is  for  training  and  evaluation. 


Comments: 


2  For  this  comparison,  you  may  have  to  imagine  instructor  workload  with  real  radio  in  the  air. 
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7)  Radio  Communications  in  Scenarios: 

Please  indicate  how  frequently  the  skills  below  rely  on  radio  communications  as  part  of 
the  scenario  context  for  team  performance.  Also,  indicate  how  important  you  believe 
the  radio  communications  simulation  is  to  training  effectiveness. 


'■  v  ' '  :v ; ' 

Importance  for  Training 

|Tyi>e-^.Skills 

Frequency 

Effectiveness 

j..  I.-.-  .  :  ■'  ■  ' 

1  2 

3  4 

5 

1  2 

3 

4  5 

i  New  ATC  Procedures  (e.g.,  PRM) 

r  *.v  ■  •••■■ '! 

Never 

Sometimes 

Often 

Very  Low 

Very  High 

Non-Routine  ATC 

1  2 

3  4 

5 

1  2 

3 

4  5 

(e.g.,  pilot/ATC  coordination) ... 

■  >  .  • 

Never 

Sometimes 

Often 

Very  Low 

Very  High 

CRM:  Crew  Coordination 
i  (e.g.,  cabin,  dispatch,  maintenance) 

■::V-  r::*  ■ 

1  2 
Never 

3  4 

Sometimes 

5 

Often 

1  2 
Very  Low 

3 

4  5 

Very  High 

;  '■ ;  ■■■■■ ;  v  ■ 

1  2 

3  4 

5 

1  2 

3 

4  5 

Distraction  Management 

Never 

Sometimes 

Often 

Very  Low 

Very  High 

■  ■■■  ,:v-  ■  ' 

1  2 

3  4 

5 

1  2 

3 

4  5 

Situation  Awareness  /  Party  Line 

Never 

Sometimes 

Often 

Very  Low 

Very  High 

1  2 

3  4 

5 

1  2 

3 

4  5 

Other: .  . . . 

: 

Never 

Sometimes 

Often 

Very  Low 

Very  High 

Comments: 
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8)  A  Few  Final  Questions... 

a)  Please  indicate  the  percentage  of  time  and  effort  you  spend: 


|fvent  „ 

Running 

Simulation 

Simulating  Radio 
Communications 

Instructing 

Observing 

Other: 

Total 

IHUi 

100% 

100% 

ns 

100% 

b)  If  the  following  radio  communications  simulations  were  automated,  what  do  you  think 
the  effect  would  be  on  instructor  workload  and  training  effectiveness? 


communSnssimulations  were 
automated* 

Instructor/Evaluator  Workload 
would  be: 

Training  Effectiveness  would  be: 

Frequency  change/  handoffs 

1  2  3  4  5 

Lower  Same  Higher 

1  2  3  4  5 

Lower  Same  Higher 

Clearances  to  own  aircraft 

1  2  3  4  5 

Lower  Same  Higher 

1  2  3  4  5 

Lower  Same  Higher 

Clearances  to  other  aircraft 

1  2  3  4  5 

Lower  Same  Higher 

1  2  3  4  5 

Lower  Same  Higher 

ACARS 

1  2  3  4  5 

Lower  Same  Higher 

1  2  3  4  5 

Lower  Same  Higher 

c)  What  other  types  of  radio  communications  would  you  like  to  see  automated,  and  what 
do  you  think  the  effect  would  be  on  instructor  workload  and  training  effectiveness? 
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9)  Please  complete  the  following  background  and  contact  information: 

Your  Simulator  Training/Evaluation  Responsibilities: 


Your  length  of  employment  in  this  position  (include  time  with  your  current  employer  as 
well  as  any  time  in  a  similar  position  with  a  previous  employer): 
.  (years,  months) 

In  case  we  need  to  contact  you  for  follow-up  questions: 

Telephone:  ( _ )  ( _ - _ ) 

E-mail,  if  available: . 


If  you  have  any  additional  comments,  please  enter  them  below. 


End  of  Survey! 

Thank  you  very  much! 
Please  mail  the  completed  form  to: 


Dr.  Alfred  T.  Lee 
Beta  Research,  Inc. 
P.O.  Box  66423 
Scotts  Valley,  CA  95067 
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